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PREFACE 


This  investigation  evaluated  the  effectiveness  of  an  out-of-the-window  tower  simulation*  by 
studying  the  first  year  of  training  conducted  in  the  simulated  tower.  The  author  wishes  to  express 
gratitude  to  the  ten  student  controllers  whose  training  was  studied  in  this  evaluation.  Their  many 
observations  provided  an  invaluable  source  of  insight  about  the  simulation. 

Jon  Bremseth,  an  O’Hare  training  specialist,  operated  the  simulation.  He  previously  developed  a 
training  laboratory  using  videotapes  correlated  to  flight  progress  strips,  setting  the  stage  for  the 
current,  interactive  simulation  training  facility.  Jon  served  as  the  facility  contact  person  for  this 
evaluation.  He  also  assisted  this  evaluation  by  ensuring  that  one  session  was  videotaped  for  each 
trainee  each  week  and  by  providing  the  corresponding  flightstrips  which  were  used  in  data 
reduction.  He  contributed  useful  observations  on  the  productive  use  of  out-of-the-window  tower 
simulation,  many  of  which  appear  in  this  report. 

Many  other  individuals  at  O’Hare  deserve  recognition  for  their  outstanding  contributions  to  this 
evaluation.  Ellen  Jaeger  is  the  Assistant  Manager  for  Training  at  O’Hare.  She  and  Roy  Hillen, 
an  O’Hare  training  specialist,  collected  preliminary  data  on  trainee  errors  during  their  simulation 
training  and  provided  information  about  the  trainees’  prior  experience.  Tower  supervisors  Bob 
Kamick  and  Tim  Fitzgerald,  and  area  manager  Kevin  Markwell  provided  ratings  of  trainee  skills. 
Matt  Dunne,  Acting  Assistant  Manager  of  Traffic  Management  provided  typical  taxi  delays,  taxi 
times  and  aircraft  rates  during  an  outbound  push.  The  author  also  recognizes  and  appreciates  the 
critical  support  provided  by  Tower  Air  Traffic  Manager,  Bill  Halleck. 

The  author  thanks  three  individuals  for  their  help  with  technical  aspects  of  this  project.  Mitch 
Grossberg  of  FAA/ ACT-500  contributed  valuable  suggestions  on  the  format  used  to  collect 
supervisor  ratings.  Patricia  Pilanen,  a  tower  training  specialist  at  Logan  Airport,  answered  many 
of  the  author’s  questions  about  tower  training  and  drew  the  author’s  attention  to  the  need  for 
ground  control  trainees  to  learn  to  manage  flight  progress  strips  without  reducing  their  attention 
to  the  airport  movement  areas.  Karl  Hergenrother  of  RSPA/Volpe  Center  created  the  data 
reduction  programs  used  to  reduce  duration  data  from  videotape,  developed  videotape  data 
reduction  procedures,  and  contributed  valuable  observations  on  the  simulation  process. 

Objective  data  reduced  from  videotape  played  a  key  role  in  this  evaluation.  This  exacting  work 
was  accomplished  by  Tufts  University  engineering  psychology  students.  Ana  Pons,  Kathleen 
Kim,  and  Trudi  Leone,  while  they  were  employed  at  RSPA/Volpe  Center.  Bill  Voss  at  ATZ-200 
provided  the  requirement  for  this  study.  Larry  Cole  at  AAR-200  sponsored  the  effort.  Their 
continuing  interest  and  support  were  critical  to  the  completion  of  this  project. 


1.  Cover  photograph  of  TowerPro™  courtesy  of  Wesson  International,  Inc, 


iii 


METRIC/ENGLISH  CONVERSION  FACTORS 


ENGLISH  TO  METRIC 


METRIC  TO  ENGLISH 


LENGTH  (APPROXIMATE) 

1  inch  (in)  =  2.5  centimeters  (cm) 
1  foot  (ft)  =  30  centimeters  (cm) 
1  yard  (yd)  =  0.9  meter  (m) 

1  mile  (mi)  =  1 .6  kilometers  (km) 


AREA  (APPROXIMATE) 

1  square  inch  (sq  in,  in2)  =  6.5  square  centimeters  (cm^) 
1  square  foot  (sq  ft,  ft^)  =  0.09  square  meter  (m2) 

1  square  yard  (sq  yd,  yd2)  =  0.8  square  meter  (m2) 

1  square  mile  (sq  mi,  mi2)  =  2,6  square  kilometers  (km2) 

1  acre  =  0.4  hectare  (ha)  =  4,000  square  meters  (m2) 


MASS  -  WEIGHT  (approximate) 

1  ounce  (oz)  =  28  grams  (gm) 

1  pound  (lb)  =  .45  kilogram  (kg) 

1  short  ton  =  2,000  pounds  (lb)  =  0.9  tonne  (t) 


LENGTH  (APPROXIMATE) 

1  millimeter  (mm)  =  0.04  inch  (in) 

1  centimeter  (cm)  =  0.4  Inch  (in) 

1  meter  (m)  =  3.3  feet  (ft) 

1  meter  (m)  =  1.1  yards  (yd) 
1  kilometer  (km)  =  0.6  mile  (mi) 


AREA  (APPROXIMATE) 

1  square  centimeter  (cm2)  -  o.16  square  inch  (sq  in,  in2) 

1  square  meter  (m2)  =  1.2  square  yards  (sq  yd,  yd2) 
1  square  kilometer  (km2)  =  0.4  square  mile  (sq  mi,  mi2) 
10,000  square  meters  (m2)  =  1  hectare  (ha)  =  2.5  acres 


MASS  -  WEIGHT  (approximate) 

1  gram  (gm)  =  0.036  ounce  (oz) 

1  kilogram  (kg)  =  2.2  pounds  (lb) 

1  tonne  (t)  =  1 ,000  kilograms  (kg)  =  1.1  short  tons 


VOLUME  (APPROXIMATE) 

1  teaspoon  (tsp)  =  5  milliliters  (ml) 

1  tablespoon  (tbsp)  =  1 5  milliliters  (ml) 

1  fluid  ounce  (fl  oz)  =  30  milliliters  (ml) 

1  cup  (c)  =  0.24  liter  (I) 

1  pint  (pt)  =  0.47  liter  (I) 

1  quart  (qt)  =  0.96  liter  (I) 

1  gallon  (gal)  =  3.8  liters  (I) 

1  cubic  foot  (cu  ft,  ft3)  =  0.03  cubic  meter  (m^) 
1  cubic  yard  (cu  yd,  yd^)  =  0.76  cubic  meter  (m^) 


TEMPERATURE  (exact) 

°C=5/9(°F  -  32) 


^  VOLUME  (APPROXIMATE) 

1  milliliter  (ml)  =  0.03  fluid  ounce  (fl  oz) 


1  liter  (I) 
1  liter  (I) 
1  liter  (I) 
1  cubic  meter  (m^) 
1  cubic  meter  (m^) 


2.1  pints  (pt) 

1.06  quarts  (qt) 

0.26  gallon  (gal) 

36  cubic  feet  (cu  ft,  ft^) 

1 .3  cubic  yards  (cu  yd,  yd^) 


TEMPERATURE  (exact) 

°F=9/5(°C)  +  32 


QUICK  INCH-CENTIMETER  LENGTH  CONVERSION 


INCHES  0 


CENTIMETERS  0  1  2  3  4  5  6  7  8  9  10  11  12  13 


QUICK  FAHRENHEIT-CELSIUS  TEMPERATURE  CONVERSION 

°F  -40°  -22°  -4°  14°  32°  50°  68°  86°  104°  122°  140°  158°  176°  194°  212° 

III  I  I  11 _ 1 _ J _ 1 _ L _ I _ I _ I - 1 — 


— I - 1  I  r  I  1  III  I  I  I  I  I  I 

°C  -40°  -30°  -20°  -10°  0°  10°  20°  30°  40°  50°  60°  70°  80°  90°  100° 


For  more  exact  and  or  other  conversion  factors,  see  NIST  Miscellaneous  Publication  286,  Units  of  Weights  and 
Measures.  Price  $2.50.  SD  Catalog  No.  Cl  3  10286. 


updated  8/1/96 


IV 


TABLE  OF  CONTENTS 


Section  Page 


1.  INTRODUCTION . 1 

1.1  Purpose . 1 

1.2  Background . .  1 

1 .3  Constraints  on  the  Evaluation . 2 

1.3.1  Lack  of  a  Concurrent  Control  Group . . . 3 

1 .3.2  Ongoing  Training . 3 

1.3.3  Sample  Size . 3 

2.  THE  SIMULATION . 5 

2.1  Description . 5 

2.2  Limitations  of  the  Simulation . . . 6 

2.2. 1  Partial  Field  of  View . 6 

2.2.2  No  Inbound  Aircraft . 8 

2.2.3  Automated  Controller  Speech  Recognition/Synthetic  Pilot  Speech . 8 

2.3  Precursor  to  Simulation  Training  at  O’Hare . 9 

3.  OUT-OF-THE-WINDOW  SIMULATION  TRAINING  METHODS . 1 1 

3.1  Trainees  and  Schedule . 1 1 

3.2  Simulation  Training  Procedure . 1 1 

3.3  Modification  of  Training  Methods . 13 

4.  EVALUATION  METHOD . 15 

4.1  Measures  of  Effectiveness . 15 

4.2  Videotaped  Training  Sessions . 15 

5.  RESULTS  AND  DISCUSSION . 17 

5.1  Post-Simulation  Training  Measures . 17 

5.1.1  Days-to-Certification . 17 

5.1.2  Hours-to-Certification . 18 

5.1.3  Tower  Supervisor  and  Area  Manager  Ratings  of  Ground  Control 

Skills . 19 

5.2  Simulation  Training  Measures . 21 

5.2.1  Preliminary  Data  Considerations  and  Analysis . 21 

5.2.2  Taxi  Delay . 23 

5.2.3  Taxi  Time . 24 

5.2.4  Window  and  Stripboard  Scan . 26 

5.2.5  Analysis  of  Instructor  Assistance . 26 


V 


TABLE  OF  CONTENTS  (continued) 

Section  Eigc 


6.  FINDINGS  AND  RECOMMENDATIONS . 29 

6.1  Training  Using  the  Simulator  was  Effective . 29 

6.2  Training  Using  the  Simulator  was  Faster  and  More  Effective  then  Traditional 

Training . 30 

6.3  Training  Using  Tower  Simulation  is  Likely  to  Show  Increased  Benefits  Upon 

Upgrading . 31 

6.4  Current  Cost  of  Recommended  Upgrading . 34 

6.5  New  Applications  of  Tower  Simulation  Technology . 34 

APPENDIX  A;  FOCUS  GROUP  SUMMARY . 39 

APPENDIX  B:  BASELINE  DEVELOPMENT  OF  GROUND  CONTROL  SKILLS . 43 

APPENDIX  C:  OUTLINE  OF  RECOMMENDED  REQUIREMENTS  FOR  A  TOWER 
SYSTEM  ASSESSMENT  SIMULATOR . 45 


vi 


LIST  OF  FIGURES 


Figure 


1.  SIMULATION  EQUIPMENT  LAYOUT . 5 

2.  PARTIAL  FIELD  OF  VIEW . 7 

3.  AUTOMATED  SPEECH  RECOGNITION/SYNTHETIC  SPEECH:  EFFECT  ON  TAXI 

DELAY . 22 

4.  AVERAGE  TAXI  DELAY . 23 

5.  TAXI  DELAY  TIMES  LESS  THAN  OR  EQUAL  TO  30  SECONDS . 24 

6.  AVERAGE  TAXI  TIME . 25 

7.  SCAN  TOWARD  WINDOW  AND  STRIPBOARD . 26 

8.  ASSISTANCE  REQUIRED  (MAJOR  CATEGORIES) . 28 

9.  INSTRUCTOR-CORRECTED  AND  SELF-CORRECTED  COMMUNICATIONS . 28 

LIST  OF  TABLES 

Table 


1 .  AIRPORT  CONFIGURATIONS  AND  DEPARTURE  RUNWAYS  AT  O’HARE 

AIRPORT . 1 1 

2.  TRAINING  OBJECTIVES . 12 

3.  DAYS-TO-CERTIFICATION . 17 

4.  HOURS-TO-CERTinCATION . 18 

5.  STRENGTH  OF  GROUND  CONTROL  SKILLS  FOLLOWING  ONE  MONTH  OF 

SIMULATION  TRAINING  COMPARED  TO  BASELINE  . 20 

6.  EQUIVALENT  MONTHS  OF  OJT:  SIMULATION  TRAINING  COMPARED  TO 

BASELINE . 21 

7.  INSTRUCTOR  ASSISTANCE  BY  WEEK  OF  TRAINING  (MEAN  NUMBER  OF 

ASSISTS) . 27 


vii 


viii 


EXECUTIVE  SUMMARY 


The  objective  of  this  evaluation  was  to  gather  evidence  bearing  on  the  potential  usefulness  of 
out-of-the- window  ATC  tower  simulation  for  training  tower  controllers.  The  FAA  Research  and 
Development  Service  (ARD)  sponsored  the  development  of  an  out-of-the-window  simulation  as 
a  proof-of-concept  demonstration.  It  was  about  to  be  used  for  the  first  time  to  train 
developmental  controllers  at  O’ Hare  International  Airport  in  Chicago.  This  simulation  was  a 
prototype  (beta)  version  of  TowerPro''"'^  (Aviation  Simulations,  Inc.).  It  had  been  developed 
specifically  to  train  outbound  ground  control  at  O’Hare.  The  data  for  this  evaluation  were 
collected  on  the  progress  and  relative  post-training  proficiency  of  all  ten  developmental 
controllers  who  completed  simulation  training  at  O’Hare  during  1994,  the  first  year  of  its  use. 

The  simulation  implemented  at  O’Hare  included  one  controller  position,  outbound  ground 
control.  An  out-of-the-window  view  was  projected  onto  three  visual  displays  which 
approximated  the  size  of  actual  tower  windows.  Computer-generated  aircraft  appeared  to  move 
in  three  dimensions  on  these  displays  at  rates  which  varied  with  the  type  of  aircraft.  The  aircraft 
were  representative  of  the  types  and  companies  at  O’ Hare.  A  session  included  30  to  90  aircraft 
in  50  minutes. 

The  simulation  was  limited  to  three  screens,  so  that  only  a  135  degree  section  of  the  airport  could 
be  displayed  at  any  one  time.  Controllers  at  O’ Hare  must  observe  aircraft  movements  on  all 
sides  of  the  tower.  In  the  simulation,  instead  of  turning  to  look  through  a  different  window,  the 
trainees  used  a  hand-held  control  to  rotate  or  “pan”  the  displayed  section  to  show  the  relevant 
part  of  the  airport.  The  simulation  was  also  limited  to  outbound  aircraft:  No  inbound  aircraft 
were  included  in  the  simulation  scenarios. 

The  evaluation  was  constrained  by  the  lack  of  a  concurrent  control  group  consisting  of 
developmental  controllers  who  would  have  received  only  traditional  training.  Instead, 
comparisons  were  made  between  the  trainees  who  used  the  simulation  and  those  who  had 
previously  been  trained  at  O’Hare,  prior  to  the  installation  of  the  simulation. 

The  primary  measures  of  simulation  effectiveness  concerned  how  long  (days  and  total  hours)  it 
took  a  developmental  controller  to  become  certified  on  the  outbound  ground  control  position. 

The  primary  measures  were  supplemented  by  ratings  made  by  tower  supervisors  on  specific 
ground  control  skills,  following  simulation  training.  These  expert  ratings  were  compared  to  the 
supervisors’  and  an  area  manager’s  recollections  of  the  baseline  strength  of  developmental 
controller  skills  after  varying  amounts  of  time. 

Additional  data  were  reduced  from  videotapes  of  the  simulation  training  sessions.  The  measures 
obtained  included  taxi  delay,  defined  as  the  time  it  took  the  ground  controller  to  provide  taxi 
instructions  to  an  aircraft  that  was  ready  to  taxi;  taxi  time,  which  began  when  an  aircraft  received 
taxi  instructions  and  ended  when  it  was  instructed  to  switch  radio  frequencies  to  a  tower 
frequency;  and  visual  scanning  directed  toward  the  “window”  or  toward  the  stripboard. 

Instructor  comments  made  during  the  videotaped  sessions  were  classified  according  to  topic  and 
tallied  to  provide  additional  indications  of  the  development  of  specific  ground  control  skills. 


IX 


Also,  the  first  group  of  trainees’  perceptions  of  the  simulation  training  were  solicited  in  a  focus 
group  session. 

The  data  collected  support  the  following  conclusions: 

The  out-of- the- window  tower  simulation  used  at  O’ Hare: 

•  is  an  effective  tool  for  training  many  ground  control  skills  and  knowledge,  and  appears 
particularly  effective  in  increasing  trainees’  working  speed. 

•  is  more  effective  for  training  outbound  ground  control  than  more  traditional  training 
techniques. 

The  effectiveness  of  the  simulation  was  indicated  by  the  following: 

I.  Trainees’  ground  control  skills  increased  noticeably  during  simulation  training. 

Taxi  delay  decreased  consistently  from  the  second  week  to  the  end  of  training. 

Stripboard  scanning  began  to  consistently  decrease  after  the  second  week  of  training,  while 
window  scanning  began  to  increase  after  the  third  week  of  training. 

The  amount  of  assistance  needed  for  a  trainee  to  properly  handle  a  scenario  decreased 
consistently  throughout  training. 

II.  Training  using  the  simulator  was  more  effective  than  traditional  training. 

The  developmental  controllers  who  were  trained  using  simulation  became  certified  on  the 
outbound  ground  control  position  in  25  %  fewer  days  than  developmental  controllers  who  were 
trained  without  simulation.  However,  using  simulation,  the  trainees  needed  about  the  same  ( only 
5%  fewer)  hours  to  become  certified  on  this  tower  position. 

Expert  ratings  of  eight  ground  control  skills  based  on  actual  tower  observations  were  all  higher 
following  simulation  training  than  following  traditional  training. 

The  difference  between  the  days-to-certification  and  total  hours-to-certification  results  can  be 
explained  in  terms  of  working  speed: 

The  Working  Speed  Hypothesis:  Simulation  training  increased  developmental 
controllers’  working  speed,  which  enabled  them  to  work  in  the  actual  tower  under  a 
wider  range  of  conditions  (i.e.,  under  heavier  or  more  complex  traffic)  and  hence  for 
more  hours  per  day  than  with  traditional  training. 
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Recommendations 

Upgrading  the  simulation  could  be  of  value  in  providing  training  for  the  following: 

1 .  Inbound  ground  control  and  coordination  between  inbound  and  outbound  ground  control 
positions 

2.  Reflexive,  correct  communications 

3.  Scanning  and  situational  awareness 

4.  Understanding  what  a  pilot  can  see  from  the  cockpit 

5.  Smooth  transition  between  window  scan  and  BRITE/ASDE  displays 

6.  Teamwork 

Simulation  training  at  O’Hare  requires  use  of  the  simulation  facility  for  roughly  six  months  per 
year.  The  most  productive  use  of  tower  simulation  at  O’ Hare  would  incorporate  enhanced 
capabilities  and  new  areas  of  application  that  would  permit  more  extensive  use.  The  following 
enhancements  and  new  application  areas  are  discussed: 

1 .  Individual  performance  enhancement  for  current  controllers 

2.  Team  performance  enhancement  for  current  controllers 

3.  Training  in  the  handling  of  unusual  situations 

4.  Optimizing  training  through  the  application  of  experimental  training  conditions 

5.  Tower  controller  candidate  screening 

6.  Assessment  of  new  tower  equipment,  procedures,  and  airport  configurations. 
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1.  INTRODUCTION 


1.1  PURPOSE 

The  purpose  of  this  effort  was  to  provide  human  factors  testing  and  evaluation  of  an  air  traffic 
control  tower  (ATCT)  out-of-the-window  (OTW)  controller  training  simulation  located  at 
O’Hare  International  Airport  in  Chicago.  It  was  intended  to  analyze  and  quantitatively  measure, 
to  the  extent  possible,  the  potential  usefulness  of  devices  of  this  type  if  they  were  to  be  installed 
at  selected  high  density  or  complex  ATC  facilities.  Goals  included: 

•  An  evaluation  of  the  impact  of  tower  simulation  on  time-to-certification  on  position 

•  Quantitative  estimates  of  any  savings  to  the  user  (airlines,  aircraft  operators) 

•  Recommendations  for  the  most  productive  application  of  OTW  simulators 

•  Recommendations  for  functional  specifications  of  future  simulators  based  on  current  systems 

•  Evaluation  of  the  potential  of  tower  simulation  for  rapid  prototyping  to  assess  the  impact  of 
new  tower  equipment,  procedures,  or  configurations. 

1.2  BACKGROUND 

The  U.S.  air  traffic  system  requires  well-trained  tower  controllers.  Controllers  receive 
training  at  the  FAA  Academy,  but  the  training  for  high  activity  (operational  level  5)  towers 
normally  follows  years  of  operational  experience  as  a  full  performance  level  controller  at  less 
active  towers,  plus  many  months  of  training  at  the  high  activity  tower.  Until  certified  on  all 
tower  positions,  trainees  are  termed  “developmental  controllers.”  Developmental  controllers  do 
not  work  independently  at  a  tower  position  until  they  have  met  the  criteria  for  certification  on 
that  position.  At  the  nations’  high  activity  towers,  the  tower  positions  typically  include  flight 
data,  clearance  delivery,  ground  control,  local  control,  and  supervisor. 

This  report  concerns  outbound  ground  control  training  at  O’Hare  Tower.  At  O’Hare,  controllers 
handle  the  largest  number  of  aircraft  operations  of  any  tower  in  the  country.  In  1992,  O’ Hare 
recorded  nearly  thirty  million  enplanements,  with  a  seventy  to  eighty  percent  increase  in 
enplanements  expected  by  2010  .  The  volume  and  complexity  of  its  traffic  make  outbound 
ground  control  at  O’Hare  arguably  the  most  challenging  position  to  work  at  and  the  most  difficult 
position  to  learn  of  any  position  at  any  air  traffic  facility  in  the  U.S. 

During  daily  periods  of  high  activity,  ground  control  responsibilities  are  divided  between  two 
positions  at  O’ Hare:  the  inbound  ground  control  position  is  responsible  for  arrivals  taxiing  in  to 
their  gates;  outbound  ground  control  handles  departures  taxiing  out  toward  a  runway.  Weather 
accounts  for  much  more  delay  than  any  other  cause,  but  according  to  the  FAA  Aviation  Capacity 

a 

Enhancement  Plan,  taxi-out  consistently  accounts  for  more  delay  than  the  other  phases  of  flight  . 
This  statistic  suggests  the  importance  of  providing  safe  and  efficient  air  traffic  services  to 
outbound  aircraft.  The  same  source  indicates  that  O’Hare  ranked  second  in  delays  of  the  55 
airports  at  which  the  FAA  collects  air  traffic  delay  statistics. 


2.  FAA.  (Sept.  1994).  Terminal  Area  Forecasts  FY  1993-2010.  FAA-APO-94-1 1. 

3.  FAA.  (1994).  Aviation  Capacity  Enhancement  Plan.  DOT/FAA/ASC-94-1. 


The  outbound  ground  control  position  at  O’ Hare  requires  considerable  knowledge  and  skill.  The 
outbound  ground  controller  must  first  decide  which  aircraft  to  call,  whether  to  provide  taxi 
instructions  or  to  make  a  traffic  call.  Determining  the  priority  of  duty  requires  almost  continuous 
attention  to  the  situation  on  the  taxiways,  particularly  during  an  “outbound  push”  (i.e.,  when 
relatively  many  aircraft  are  departing).  At  the  same  time,  he  or  she  must  maintain  an  awareness 
of  the  location  of  aircraft  that  are  ready  to  taxi.  This  controller  decides  which  one  of  O’Hare’s 
eight  departure  runways  to  send  each  departing  aircraft  to,  and  in  what  sequence,  decisions  that 
must  take  into  account  the  type  of  aircraft  and  its  initial  route  of  flight.  Then  the  outbound 
ground  controller  must  decide  which  series  of  taxiways  the  aircraft  should  use  to  reach  its 
runway.  After  transmitting  the  departure  runway  and  taxi  route  to  the  pilot,  ground  control  must 
make  certain  that  the  pilot  understood  the  instructions,  and  follows  the  taxi  route.  Often  the 
outbound  ground  controller  needs  to  coordinate  with  the  inbound  ground  controller  and  local 
controllers  when  the  aircraft  is  expected  to  enter  their  areas  of  responsibility. 

The  outbound  ground  controller  has  typically  required  more  than  a  year  of  traditional  on-the-job 
training  (OJT)  before  a  supervisor  certifies  his  or  her  ability  to  work  independently  at  this 
position.  During  this  time,  an  instructor  must  supervise  the  developmental  controller  (trainee) 
while  the  trainee  controls  aircraft.  At  times,  the  situation  on  the  airport  surface  is  too  busy  for 
the  trainee  to  handle,  requiring  the  instructor  to  take  over.  At  other  times,  there  is  too  little  traffic 
or  complexity  for  the  trainee  to  increase  the  level  of  his  or  her  skills.  The  ideal  training  situation 
is  one  with  enough  traffic  and  complexity  to  allow  the  trainee  to  gain  confidence  in  handling 
difficult  situations,  but  also  one  that  is  not  so  difficult  that  safety  or  the  expeditious  flow  of 
aircraft  could  be  reduced.  The  ideal  training  situation  does  not  occur  often  in  actuality,  so  OJT 
proceeds  more  slowly  than  if  the  ideal  training  situation  were  always  available. 

In  an  effort  to  improve  the  speed  and  quality  of  training,  the  training  specialists  at  O’ Hare  were 
provided  a  prototype  tower  controller  training  simulator,  TowerPro™  (Aviation  Simulations, 
Inc.).  This  prototype  was  developed  for  a  proof-of-concept  demonstration  of  its  potential  for 
training  outbound  ground  control  at  O’Hare.  The  simulation  operator,  who  was  also  a  training 
specialist,  prepared  simulation  scenarios  for  training  on  what  he  believed  would  provide  the  ideal 
amount  of  traffic  and  complexity  for  each  trainee.  Simulation  thus  permitted  the  training  of 
ground  controllers  to  always  proceed  under  planned  conditions.  To  the  extent  that  this  training 
was  sufficiently  realistic  to  transfer  to  performance  in  the  tower,  simulation  training  was 
expected  to  reduce  training  time,  particularly  for  the  outbound  ground  control  position. 

1.3  CONSTRAINTS  ON  THE  EVALUATION 

This  evaluation  was  conducted  as  a  field  study.  As  such,  it  was  limited  in  the  use  of  statistical 
control  groups  and  independent  variables.  This  section  describes  these  constraints  and  their 
impact  on  the  evaluation. 

•  Lack  of  a  concurrent  control  group 

•  Ongoing  training 

•  Sample  size 
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1.3.1  Lack  of  a  Concurrent  Control  Group 


A  constraint  on  the  evaluation  was  the  lack  of  a  concurrent  control  group  consisting  of 
developmental  controllers  who  would  have  received  only  traditional  training.  No  concurrent 
control  group  was  included  in  the  investigation  because  it  was  not  considered  ethical  to  withhold 
what  was  expected  to  be  better  training  from  a  control  group.  A  possible  alternative  involved 
comparisons  of  simulation  training  at  O’Hare  to  traditional  training  at  another  tower.  However, 
the  results  of  such  comparisons  would  be  suspect  due  to  confounding  with  other  differences 
between  the  two  towers.  Instead,  comparisons  were  made  between  the  trainees  who  used  the 
simulation  and  those  who  had  previously  been  trained  at  O’Hare,  prior  to  the  installation  of  the 
simulation. 

1.3.2  Ongoing  Training 

The  data  gathered  for  this  study  were  collected  during  the  first  year  the  training  staff  at  O’Hare 
used  an  OTW  tower  simulator  to  train  tower  controllers.  Three  groups  of  controllers  received 
training  over  the  course  of  this  one-year  study.  The  O’Hare  training  staff  gave  eonsiderable 
thought  to  the  way  they  used  the  simulator  and  modified  their  training  practices  in  an  attempt  to 
improve  the  training  of  each  successive  group.  As  a  result,  the  training  methods  used  for  the 
three  groups  differed.  This  led  to  instances  where  it  appeared  inadvisable  to  combine  the  training 
data  obtained  from  the  different  groups. 

Another  constraint  on  the  evaluation  resulted  from  the  increasingly  difficult  demands  of  the 
training  program.  Simply  stated,  more  skills  were  required  at  the  end  of  training  than  at  the  start 
(see  section  3.2).  Due  to  this  inereasing  demand,  the  trainees  needed  to  increase  their  skills  for 
their  performance  to  remain  at  the  same  level.  This  constraint  complicated  the  interpretation  of 
data  obtained  from  simulation  training  sessions. 

1.3.3  Sample  Size 

This  evaluation  was  based  on  the  performance  of  all  ten  developmental  controllers  who  received 
simulation  training  during  1994  at  O’Hare  Tower.  The  small  size  of  this  sample  precluded 
conducting  statistical  tests  on  many  of  the  differences  observed. 
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2.  THE  SIMULATION 


2.1  DESCRIPTION 

The  training  simulation  evaluated  was  TOWER/Pro^M,  a  product  of  Aviation  Simulations,  Inc. 

It  cost  approximately  $500,000.  It  was  set  up  in  an  O’Hare  Tower  training  room  as  shown  in 
Figure  1 .  There,  it  was  used  to  train  performance  on  the  outbound  ground  control  position.  The 
three  visual  displays  showed  aircraft  moving  about  the  simulated  O’Hare  Airport.  The  displays 
were  approximately  the  size  of  actual  O’Hare  Tower  cab  windows,  although  they  were 
partitioned  somewhat  differently.  Because  it  presented  the  airport  in  a  manner  virtually  identical 
to  the  airport  as  seen  through  the  actual  tower  windows  (but  see  the  field  of  view  limitation 
described  in  section  2.2.1),  the  device  is  considered  an  “out-of-the-window”  tower  simulator. 
The  simulation  showed  the  entire  airport  including  all  taxiways  and  runways,  permitting  practice 
with  all  runway  configurations.  The  computer-generated  aircraft  appeared  to  move  in  three 
dimensions  at  rates  which  depended  upon  the  type  of  aircraft.  Aircraft  crossed  from  display  to 
display  without  interruption,  much  as  planes  seen  from  the  tower  cab  appear  to  move  along 
taxiways  from  window  to  window.  The  simulation  included  a  representative  mix  of  air  carriers 
and  propeller-driven  types  of  aircraft.  The  airport  was  displayed  as  if  seen  from  slightly  above 
the  tower  cab  because  the  simulation’s  airport  image  was  constructed  from  photographs  taken 
from  the  roof  of  the  cab. 


Window  Display  2 


FIGURE  1.  SIMULATION  EQUIPMENT  LAYOUT 
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The  airport  and  aircraft  images  were  projected  from  a  ceiling-mounted  source  that  was  separated 
by  a  curtain  from  the  rest  of  the  training  room.  The  ambient  lighting  in  the  training  room  was 
maintained  at  a  dim  level  while  the  simulation  was  running  to  minimize  reductions  in  color 
contrast  on  the  window  displays.  Accurate  color  representation  is  important  because  the  colored 
logos  and  markings  on  aircraft  provide  cues  to  an  aircarrier’s  company,  and  controllers  use  them 
to  identify  aircraft. 

An  instructor  and  the  simulation  operator  provided  training  in  the  simulated  O’ Hare 
environment.  Each  instructor  was  assigned  to  a  pair  of  developmental  controllers,  and  instructed 
them  during  simulation  training  and  during  OJT.  While  one  trainee  assumed  the  role  of 
outbound  ground  controller,  the  other  served  as  a  “ground  meterer.”  The  ground  meterer’s  role 
in  the  simulation  was  to  set  up  the  flight  strip  for  each  flight  as  it  was  about  to  enter  the 
simulation. 

During  the  simulation,  the  trainee  stood  behind  a  flight  progress  strip  bay  facing  the  three 
screens.  He  (all  trainees  during  the  one-year  data  collection  period  were  male)  wore  a  standard 
headset  for  communication  with  simulation  “pilots.”  A  camcorder  used  to  collect  data  for  this 
study  was  located  on  the  trainee’s  left  and  was  focused  on  the  trainee  to  record  his  voice  and  the 
direction  of  his  gaze.  The  ground  meterer  stood  to  the  trainee’s  left  near  a  computer  display 
which  showed  the  airport,  current  aircraft  positions,  and  the  simulation  clock.  The  instructor 
stood  to  the  right  and  behind  the  trainee.  The  simulation  operator,  who  was  also  a  training 
specialist,  sat  behind  the  instructor  at  a  console  with  aircraft  movement  controls  and  a  display 
showing  the  airport  and  aircraft  locations.  The  trainee  could  see  neither  the  ground  meterer’s  nor 
the  simulation  operator’s  display. 

2.2  LIMITATIONS  OF  THE  SIMULATION 

Every  simulation  embodies  limitations.  Following  are  the  limitations  of  the  simulation  upon 
which  this  report  is  based: 

•  partial  field  of  view 

•  automated  controller  speech  recognition/synthetic  pilot  speech 

•  no  inbound  aircraft. 

2.2.1  Partial  Field  of  View 

O’Hare  Tower  overlooks  the  airport  on  all  sides,  permitting  controllers  to  view  all  of  the 
taxiways  and  runways,  which  are  found  on  all  sides  of  the  tower.  Since  the  ground  and  local 
controllers  must  visually  locate  each  aircraft  before  providing  it  with  movement  instructions  and 
clearances,  their  scanning  must  include  every  direction.  The  simulation  could  display  all  of  the 
airport,  but  only  135  degrees  of  arc  was  visible  at  a  time  through  the  three  window  displays.  At 
O’Hare,  the  ground  controllers  often  move  about  the  tower  cab  for  a  better  view  of  the  airport.  In 
the  simulation,  though,  the  trainees  had  to  select  one  135  degree  segment  to  view  at  a  time 
(Figure  2).  They  used  a  handheld  control  similar  to  a  computer  mouse  to  rotate  or  “pan”  the 
view  until  the  desired  segment  of  the  airport  could  be  seen.  Trainees  could  vary  the  panning  rate 
from  approximately  45  degrees  per  second  to  150  degrees  per  second.  Panning  from  a  fixed 
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position  thus  took  the  place  of  moving  around  the  tower.  Thus,  trainees  needed  to  learn  an 
artificial  simulator  skill  (panning)  to  participate  in  simulation  training. 


FIGURE  2.  PARTIAL  FIELD  OF  VIEW 


It  was  probably  more  difficult  for  the  trainees  to  locate,  identify,  and  track  aircraft  in  the 
simulation  than  in  the  real  tower  because  they  needed  to  pan  the  displayed  135  degree  segment  of 
the  airport  to  the  part  that  they  wanted  to  see.  Some  evidence  of  this  difficulty  appeared  in  a 
focus  group  conducted  with  the  first  group  of  trainees  (see  Appendix  A  for  a  summary).  When  a 
controller  scans  the  airport  from  the  real  tower,  he  or  she  physically  changes  his  or  her  view  with 
eye  movements,  head  turns,  and  body  movement.  The  perceptual  system  adjusts  to  produce  a 
stable  view,  a  phenomenon  known  as  position  constancy".  In  the  simulation,  when  the  trainees 
panned  the  view  to  bring  a  target  onto  the  display,  display  motion  was  not  accompanied  by  self- 
motion,  so  relatively  fast  panning  produced  an  unstable  view.  The  effect  of  panning  was  that  the 
trainees  could  not  identify  aircraft  while  panning  quickly,  but  instead  needed  to  either  pan  slowly 
or  wait  until  the  display  stopped. 


4.  Matlin,  M.W.  (1988).  Sensation  and  Perception  (2nd  Ed.).  Allyn  and  Bacon. 
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2.2.2  No  Inbound  Aircraft 


Inbound  aircraft  were  not  simulated.  Training  focused  on  the  outbound  ground  control  position 
and  thus  required  a  maximum  of  practice  handling  outbound  aircraft.  Limiting  the  simulation  to 
inbound  aircraft  also  maintained  the  complexity  of  the  traffic  at  levels  appropriate  for  new 
developmental  controllers. 

2.2.3  Automated  Controller  Speech  Recognition/Synthetic  Pilot  Speech 

The  simulation  included  automated  speech  recognition  and  synthetic  speech  capabilities.  Speech 
recognition  was  intended  to  accept  eorrectly  phrased  taxi  instructions  and  then  respond 
automatically  with  the  corresponding  aircraft  movements  and  an  automated  pilot  readback  using 
synthetic  speech.  The  automated  system  was  available,  but  its  use  was  discontinued  because  it 
hindered  training  by  plaeing  several  artificial  constraints  on  trainee  speeeh.  As  a  result,  speech 
recognition  was  discontinued  during  the  third  week  of  attempted  use  by  the  first  group  of  trainees 
(see  Appendix  A  for  trainee  observations  regarding  speech  recognition).  The  following 
constraints  were  observed: 

1 .  Trainees  needed  to  remember  to  pause  at  the  end  of  each  instruction,  and  to  not  pause  in  the 
middle  of  the  instruction.  This  often  interfered  with  a  controller’s  speech  cadence. 

2.  It  was  impractical  to  train  the  speech  recognition  system  to  recognize  the  complete  lexicon 
used  by  ground  controllers  to  assist  pilots.  If  the  automated  system  failed  to  understand  an 
instruction,  the  utterance  did  not  elicit  a  simulator  response.  As  an  example,  it  did  not 
recognize  both  grouped  and  ungrouped  numbers  in  aircraft  callsigns.  Also,  unlike  real  pilots, 
the  simulation  did  not  recognize  common  taxi  instructions  which  identified  the  relative 
position  of  aircraft,  such  as  “Join  alfa  behind  the  second  MD80  on  your  left.”  When  a 
recognition  failure  occurred,  the  trainee  needed  to  rephrase  and  re-issue  the  instruction,  which 
slowed  and  disrupted  performance. 

3.  The  speech  recognition  system  artificially  limited  the  ways  a  trainee  could  issue  instructions 
to  multiple  aircraft  because  each  aircraft  needed  to  respond  before  another  could  accept  an 
instruction.  This  limitation  interfered  with  training  on  the  effective  use  of  the  ground  control 
frequency. 

4.  Trainees  needed  to  “train”  the  speech  recognition  system  to  recognize  their  voices,  requiring 
several  days  that  were  then  not  available  for  training. 

5.  Synthetie  pilot  responses  (readbacks  and  aircraft  movements)  occurred  more  slowly  than  in 
actual  operations. 


8 


2.3  PRECURSOR  TO  SIMULATION  TRAINING  AT  O’HARE 


In  1989,  the  O’Hare  training  staff  began  to  use  a  large  screen  television  to  display  a  videotape  of 
the  airport  taken  with  an  8  mm  camera.  Flight  progress  strips  were  correlated  to  the  videotape. 
This  non-interactive  training  aid  was  used  successfully  to  train  developmental  controllers  to  issue 
taxi  instructions,  sequence  flight  progress  strips,  and  make  traffic  calls.  This  precursor  of 
simulation  training  was  not  interactive.  The  training  specialist  could  pause  the  videotape  to  ask 
what  the  trainee  would  do  next  or  how  he  or  she  would  handle  a  particular  situation,  but  the 
trainee  could  not  alter  the  flow  of  traffic.  Each  group  that  was  trained  using  this  laboratory  aid 
completed  a  post-training  evaluation.  Many  respondents  found  the  video  “lab”  to  be  very 
helpful. 
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3.  OUT-OF-THE-WINDOW  SIMULATION  TRAINING  METHODS 


3.1  TRAINEES  AND  SCHEDULE 

Ten  developmental  controllers  completed  simulation  training  at  O’ Hare  during  the  one-year  data 
collection  period.  Seven  of  them  later  became  certified  on  outbound  ground  control.  They 
participated  in  three  groups.  The  first  two  groups  each  consisted  of  four  trainees,  and  the  third 
group  initially  consisted  of  three,  one  of  whom  was  required  to  interrupt  simulation  training  for 
medical  reasons.  He  completed  simulation  training  after  the  data  collection  period  had  ended. 

Of  the  seven  who  became  certified  on  outbound  ground  control,  three  were  in  Group  1 ,  three 
were  in  Group  2,  and  one  was  in  Group  3. 

Each  trainee  participated  in  the  simulation  as  the  outbound  ground  controller  for  one  hour  daily, 
four  days  per  week.  The  trainees  worked  in  pairs.  One  pair  participated  for  two  hours  in  the 
morning,  and  the  other  pair  participated  in  the  afternoon.  While  one  pair  was  in  simulation 
training  the  other  received  OJT  in  the  tower.  At  first  OJT  consisted  of  work  at  the  clearance 
delivery  and  flight  data  positions.  As  the  trainees’  skills  advanced,  they  began  OJT  at  the  ground 
and  local  control  positions. 

3.2  SIMULATION  TRAINING  PROCEDURE 

Prior  to  each  session,  the  simulation  operator  prepared  an  aircraft  schedule  including  the  type  and 
company  of  each  outbound  plane  and  the  time  and  gate  where  it  would  enter  the  simulation.  The 
aircraft  were  one  of  two  components  in  each  simulation  scenario.  The  second  component  was 
the  Plan,  which  corresponded  to  an  airport  configuration.  The  Plan  indicated  the  active  runways 
that  the  trainee  was  to  assign  to  the  aircraft,  depending  upon  their  type,  company,  and  gate.  At 
the  start  of  a  session,  the  simulation  operator  announced  the  Plan  that  was  to  be  applied.  See 
Table  1  for  a  list  of  the  Plans  in  use  at  O’Hare  during  the  year  data  were  collected. 


TABLE  1.  AIRPORT  CONFIGURATIONS  AND  DEPARTURE  RUNWAYS  AT 

O’HARE  AIRPORT 


Plan  (Configuration) 

Departure  Runways 

Weird 

32LatT10,  22L 

X 

32L  at  TIO,  4L,  9L,  some  32R,  14L,  32L  full  length 

B 

22L,  27L 

14’s 

9L,  9R,  27L 

27’s 

32LatT10,  32R 

9’s 

4L,  9R 

Modified  X 

4L,  9R,  32R 

14R/9R 

9L,  22L,  some  14L 
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Also  prior  to  each  session,  a  paper  flight  strip  was  printed  for  each  aircraft  showing  its  callsign, 
initial  route  of  flight,  gate,  and  the  time  it  was  scheduled  to  appear  at  its  gate.  The  flight  strips,  in 
standard  plastic  holders,  were  arrayed  on  the  ground  meterer  s  table  in  the  order  that  the  aircraft 
were  scheduled  to  enter  the  simulation.  After  the  session  began,  the  ground  meterer  monitored 
the  simulation  time  on  his  display.  When  an  aircraft  was  scheduled  to  enter  the  simulation,  he 
set  its  flight  strip  in  the  trainee’s  (controller’s)  strip  bay. 

An  aircraft  was  ready  to  taxi  in  response  to  the  simulation  operator  s  movement  commands  after 
it  automatically  pushed  back  from  its  gate  and  moved  toward  the  taxiways.  The  controller 
trainee  called  aircraft  and  issued  taxi  instructions  through  the  headset  push-to-talk  microphone 
while  marking  and  moving  the  plane’s  flight  strip.  The  simulation  operator  then  responded  as 
the  pilot  of  the  aircraft,  reading  back  instructions  and  taxiing  the  aircraft.  The  trainee  continued 
to  issue  verbal  instructions  guiding  each  aircraft  along  the  taxiways  until  he  instructed  the  pilot  to 
switch  radio  frequencies  to  a  tower  frequency.  As  the  trainee  worked  the  simulated  outbound 
traffic,  the  instructor  and  simulation  operator  responded  to  his  decisions  with  suggestions, 
questions,  and  comments.  The  sessions  lasted  approximately  50  minutes  and  were  followed  by  a 
brief  review. 

In  the  simulation,  the  simulation  operator  (a  training  specialist)  and  instructor  asked  questions, 
made  suggestions,  and  commented  on  the  trainee’s  decisions.  The  simulation  operator  also 
responded  as  a  pilot  (actually,  as  all  of  the  pilots).  In  the  pilot  role,  he  provided  verbal  readbacks 
and  acknowledgments,  taxied  the  aircraft  and,  following  the  handoff  to  a  tower  frequency,  taxied 
the  aircraft  to  the  assigned  runway  and  made  it  depart.  The  simulation  operator  paused  the 
simulation  when  extended  discussion  appeared  necessary.  The  simulation  was  paused 
infrequently  (once  or  twice  in  a  typical  session);  pauses  lasted  less  than  two  minutes. 

As  the  trainees’  proficiency  increased,  they  were  expected  to  demonstrate  more  ground  control 
skills  and  to  handle  more  aircraft.  Session  objectives  became  more  complex  with  the  progressive 
addition  of  more  advanced  skills,  as  shown  in  Table  2.  The  rate  at  which  aircraft  appeared  in  the 
scenarios  was  gradually  increased  from  one  aircraft  every  two  minutes  (30  total)  to  one  every 
forty-five  seconds  (80  total).  The  airport  configuration  or  Plan  was  changed  daily,  but  Plan 
Weird  and  Plan  B  were  used  more  often  in  training  because  they  were  more  frequently 
encountered  in  actual  operations.  A  change  of  configuration  (e.g.,  Plan  27’s  to  Plan  Weird)  was 
practiced  during  the  last  week  of  training. 


TABLE  2.  TRAINING  OBJECTIVES 


Phase  of  Training 

Objectives 

Early 

Issue  correct  taxi  routes,  master  stripboard  management 

Middle 

Assign  appropriate  runway,  issue  correct  taxi  instructions,  master 
stripboard  management 

Late 

Assign  appropriate  runway,  issue  correct  taxi  instructions,  master 
stripboard  management,  make  traffic  calls,  handle  configuration  changes 
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3.3  MODinCATION  OF  TRAINING  METHODS 


As  stated  earlier,  this  study  was  based  on  data  gathered  during  the  first  year  the  training  staff  at 
O’Hare  used  an  OTW  tower  simulator  to  conduct  training.  They  modified  some  training 
techniques  over  the  course  of  the  year.  For  example,  whereas  the  first  two  of  the  three  groups  of 
trainees  obtained  practice  on  all  O’Hare  air  carriers  throughout  their  training,  the  training 
methods  used  for  the  third  group  emphasized  one  half  of  the  airport  at  a  time  by  first 
concentrating  on  the  American  Airlines  rush.  One  anticipated  benefit  of  this  training  strategy 
was  that  it  would  reduce  the  amount  of  panning  required  and  thus  reduce  any  distration  panning 
might  have  created. 

A  set  amount  of  training  was  planned  for  the  first  two  groups.  In  contrast,  the  third  group  was 
given  training  until  a  criterion  was  reached.  As  a  result,  one  Group  3  developmental  controller 
received  four  weeks  of  simulation  training,  while  the  other  received  this  training  for  three  weeks. 
The  criterion  consisted  of  a  demonstration  that  the  trainee  could: 

-  issue  correct  taxi  routes 

-  effectively  manage  the  stripboard 

-  assign  the  appropriate  runway 

-  issue  correct  taxi  instructions 

-  make  traffic  calls  when  needed. 

The  simulation’s  automated  speech  recognition/pilot  response  system  was  used  only  during  the 
first  three  weeks  of  the  first  group’s  four-week  simulation  training  (see  section  2.2.3).  The 
second  and  third  groups  received  simulation  training  without  automated  speech 
recognition/synthetic  speech.  Group  2  received  one  week  more  simulation  training  than  Group  1 
because  it  was  not  necessary  to  take  away  the  time  needed  to  “train”  the  automated  speech 
recognition  system  from  simulation  training. 
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4.  EVALUATION  METHOD 


4.1  MEASURES  OF  EFFECTIVENESS 

This  report  describes  an  evaluation  of  the  first  year  of  training  using  an  OTW  simulator  to  teach 
the  skills  required  to  perform  outbound  ground  control  at  O’Hare  Tower.  Since  an  important 
goal  of  using  the  tower  simulation  in  training  was  to  reduce  training  time,  the  primary  evidence 
used  to  evaluate  its  effectiveness  was  the  time  required  for  the  simulation-trained 
controllers  to  become  certified  on  outbound  ground  control,  compared  to  facility  records. 
To  augment  these  data,  the  perceptions  of  tower  supervisors  were  used  to  compare  simulation- 
trained  to  traditionally  trained  developmental  controllers  on  a  variety  of  ground  control  skills. 
Their  ratings  helped  to  identify  the  specific  ground  control  skills  that  simulation  training 
benefited  and  the  strength  of  these  skills  following  simulation  training. 

The  evaluation  was  also  based  on  objective  evidence  reduced  from  videotapes  of  the  simulation 
training  sessions.  Three  types  of  evidence  were  reduced  from  the  videotapes: 

•  Taxi  Delay:  how  long  the  pilot  waited  before  receiving  taxi  instructions 

•  Taxi  Time:  how  long  aircraft  were  under  the  control  of  the  ground  controller 

•  Scan:  the  proportion  of  time  a  trainee’s  gaze  was  directed  toward  the  window  displays  or 
toward  the  flight  progress  strip  bay. 

Instructor  comments  found  on  the  videotapes  were  quantitatively  analyzed  to  provide  additional 
indications  of  the  development  of  specific  ground  control  skills  and  to  gain  a  better 
understanding  of  the  simulation  training  process. 

4.2  VIDEOTAPED  TRAINING  SESSIONS 

Taxi  time,  taxi  delay,  scanning,  and  instructor  comment  data  were  reduced  from  videotapes  of 
training  sessions.  One  training  session  for  each  developmental  controller  was  videotaped  each 
week  for  later  analysis.  The  videotaped  sessions  were  conducted  as  normal  simulation  training 
sessions.  Some  exceptions  were  made  for  the  purposes  of  the  analysis:  The  same  Plan  (Plan  X) 
was  used  in  the  videotaped  sessions  during  the  first  and  last  weeks  of  simulation  training  for 
Group  1 .  The  same  Plan  (Plan  Weird)  and  the  same  aircraft  rate  (1.5  per  minute)  was  used  in  all 
of  the  second  group’s  videotaped  sessions. 

The  training  staff  increased  the  aircraft  rate  for  Group  1  each  week  during  the  four  week 
simulation  training  period.  The  rate  increased  from  .5  aircraft  per  minute  during  the  first  week  to 
2  aircraft  per  minute  during  the  last  week.  The  videotaped  problems  for  Group  2  all  included  90 
aircraft  and  used  “Plan  Weird,’’  one  of  the  two  most  common  configurations  of  the  eight 
configurations  used  at  O’Hare  (the  other  is  Plan  X).  The  roughly  90  aircraft  per  hour  rate 
approximates  the  departing  aircraft  rate  in  an  actual  outbound  push.  These  parameters  were  kept 
constant  in  the  recorded  sessions  for  the  purpose  of  this  investigation.  Training  specialists 
determined  the  parameters  used  in  the  regular  training  sessions,  depending  upon  the  trainee’s 
skill  level  and  particular  training  needs. 
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5.  RESULTS  AND  DISCUSSION 


5.1  POST-SIMULATION  TRAINING  MEASURES 

5.1.1  Davs-to-Certification 

The  time  (i.e.,  the  number  of  days  and  total  hours)  it  took  developmental  controllers  to  become 
certified  to  work  independently  at  the  outbound  ground  control  position  was  the  primary  measure 
used  in  this  study.  Days-to-certification  was  measured  from  the  date  a  developmental  controller 
began  working  full  time  in  the  tower,  having  completed  simulator  training,  to  the  date  a  tower 
supervisor  certified  him  or  her  on  outbound  ground  control.  The  hypothesis  was  that  the 
simulation-trained  developmental  controllers  would  take  fewer  days  than  non-simulation-trained 
developmental  controllers  to  become  certified.  The  test  of  this  hypothesis  involved  a  comparison 
between  simulation-trained  controllers  and  non-simulation-trained  controllers  on  days-to- 
certification  on  outbound  ground  control.  The  non-simulation  or  “traditionally  trained”  group 
had  received  four  weeks  of  training  in  the  videotape  lab  described  in  section  2.3.  Their  days-to- 
certification  was  also  measured  from  the  date  they  began  working  full  time  in  the  tower. 

Only  data  from  those  non-simulation-trained  controllers  who  began  training  after  January  1,  1993 
were  included  in  the  no-simulation  group.  Prior  to  January,  1993,  O’Hare  certified  controllers  on 
the  outbound  and  inbound  ground  control  positions  together.  Since  simulator  training  was  only 
given  on  outbound  ground  control,  the  days  taken  for  certification  on  that  position  alone  were  the 
most  relevant.  Thus,  the  seven  controllers  who  started  training  in  or  after  January,  1993,  were 
compared  on  days-to-certification  to  the  seven  controllers  who  became  certified  on  outbound 
ground  control  following  simulator  training®.  This  comparison  is  illustrated  in  Table  3. 


TABLE  3.  DAYS-TO-CERTIFICATION 


Days-to-Certification  | 

N 

Mean 

Standard  Deviation 

Simulation 

7 

148.57 

49.65 

No  Simulation 

7 

198.71 

52.52 

Table  3  shows  that  the  simulation-trained  developmental  controllers  took  50.1  days  less,  a  25.2 
%  reduction  in  the  number  of  days-to-certification  on  outbound  ground  control.  The  difference  is 
statistically  significant,  t  (12)  =  1.84  (p  <  .05),  in  a  one-tailed  test.  This  finding  supports  the 
hypothesis  that  developmental  controllers  who  received  simulation-training  take  fewer  days  to 
become  certified  on  the  outbound  ground  control  position  than  non-simulator  trained 
developmental  controllers. 


5.  Three  developmental  controllers  were  certified  on  outbound  ground  control  following  the  year  in  which  the  data 
were  collected.  Including  this  data  would  have  produced  a  mean  of  162.6  days  and  a  standard  deviation  of  49.85 
days. 
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5.1.2  Hours-to-Certification 


The  second  part  of  time-to-certification  is  the  total  number  of  hours  required.  The  hypothesis 
was  that  the  developmental  controllers  who  received  simulation-training  would  take  fewer  hours 
to  become  certified  than  non-simulator  trained  developmental  controllers.  The  test  of  this 
hypothesis  involved  a  comparison  between  the  same  two  groups  of  developmental  controllers 
that  were  compared  in  section  5.1.1. 

Table  4  gives  the  number  of  hours  simulation-trained  developmental  controllers  required  for 
certification  on  outbound  ground  control  and  the  hours-to-certification  required  by  trainees 
without  simulator  training.  They  are  approximately  equal  (a  Mann-Whitney  test^  failed  to  reveal 
a  significant  difference,  p  >  .05).  Therefore,  simulation  training  was  effective  in  increasing 
trainees’  skills  to  a  point  where  the  trainees  could  acquire  the  needed  amount  of  OJT  in  fewer 
days,  but  it  did  not  appear  to  reduce  the  total  amount  of  OJT  needed. 

TABLE  4.  HOURS-TO-CERTIFICATION 


Hours-to-Certification 

N 

Mean 

Standard  Deviation 

Simulation 

7 

84.93 

21.18 

No  Simulation 

7 

89.42 

12.53 

It  will  be  recalled  from  section  1 .2  that  when  the  density  of  traffic  requires  a  faster  working  speed 
than  a  developmental  controller  can  produce,  the  instructor  must  take  over  the  position.  The 
developmental  controller  then  loses  an  opportunity  to  increase  his  or  her  ground  control  skills 
through  practice.  Accordingly,  a  hypothesis  that  would  explain  the  time-to-certification  results  is 
that  simulation  training  enables  trainees  to  work  more  rapidly  and  thus  to  handle  higher  traffic 
densities  in  OJT.  This  can  be  called  the  working  speed  hypothesis. 

The  working  speed  hypothesis  suggests  that  with  simulation  training,  developmental  controllers 
can  obtain  more  hours  of  OJT  per  day,  resulting  in  a  need  for  fewer  days  of  OJT  than  without 
simulation  training.  That  simulation  training  did  not  appear  to  affect  hours-to-certification 
suggests  that  simulation  training  does  not  address  at  least  some  critical  ground  control  skills. 
These  skills  apparently  develop  independent  of  working  speed  so  that  an  increased  working 
speed  does  not  reduce  the  number  of  hours  of  OJT  needed  to  develop  these  other  skills.  The 
post-simulation-training  supervisor  ratings  presented  in  the  following  section  (5.1.3)  and  the 
training  session  measures  presented  in  section  5.2.5  provide  evidence  related  to  the  identification 
of  these  skills. 


6.  This  nonparametric  test  was  used  because  the  homogeneity  of  variance  assumption  underlying  the  more  powerful 
parametric  t-test  was  not  met,  as  indicated  by  the  standard  deviation  data  presented  in  Table  4. 
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5.1.3  Tower  Supervisor  and  Area  Manager  Ratings  of  Ground  Control  Skills 


The  measures  of  time-to-certification  provide  overall  indices  of  the  benefits  that  can  result  from 
simulation  training,  but  these  measures  do  not  indicate  the  effect  of  simulation  training  on 
specific  skills.  Supervisory  (expert)  rating  data  were  obtained  to  permit  an  examination  of  the 
skills  exhibited  during  initial  outbound  ground  control  performance  following  simulation 
training.  The  strengths  of  these  skills  were  compared  to  a  baseline  that  represents  the 
development  of  ground  control  skills  without  simulation  training. 

The  baseline  was  constructed  from  ratings  of  typical  OJT  progress  for  traditionally  trained 
average  and  fast  learners  on  eight  categories  of  ground  control  skill.  An  O’ Hare  Tower  area 
manager  and  two  O’Hare  Tower  supervisors  made  the  baseline  ratings.  Each  of  the  raters  had 
worked  as  O’Hare  instructors,  area  manager,  or  supervisors  for  at  least  six  years.  Thus,  all  had 
evaluated  ground  control  skills  at  O’Hare  prior  to  the  use  of  simulation  training. 

The  baseline  was  constructed  from  the  supervisors’  and  area  manager’s  recollections.  This 
procedure  was  chosen  because  it  was  not  possible  to  obtain  the  baseline  ratings  from  direct 
observation.  For  each  skill  category,  the  supervisors  estimated  the  strength  of  the  skill  for  a 
developmental  controller  with  one  week  to  12  months  experience  in  the  tower.  They  used  five- 
point  scales  to  make  their  ratings  in  each  of  the  eight  skill  areas.  The  mean  baseline  ratings  for 
“average  learners”  and  “fast  learners”  and  the  meaning  of  each  point  on  each  of  the  5-point  scales 
appear  in  Appendix  B  . 

The  supervisors  were  asked  to  provide  ratings  on  the  same  eight  skills  for  the  simulation-trained 
developmental  controllers  following  each  group’s  completion  of  simulation  training.  The  ratings 
for  a  trainee  were  to  be  made  as  soon  as  possible  following  sufficient  observation  of  his  work  in 
the  tower.  In  all  cases  they  were  completed  within  three  weeks  following  the  end  of  simulation 
training  for  the  individual,  after  the  supervisor  had  observed  the  trainee’s  ground  control  skills 
for  at  least  one  hour.  The  ratings  for  Group  1  were  made  two  weeks  after  the  supervisors  and 
area  manager  provided  the  baseline  data. 

Table  5  presents  the  rated  strengths  of  the  eight  skill  categories  following  one  month  of 
simulation  training  compared  to  the  strengths  of  the  same  skills  following  one  month  of 
traditional  training.  The  table  presents  the  mean  ratings  for  the  seven  developmental  controllers 
who  became  certified  on  outbound  ground  control.  Table  6  shows  how  many  months  of  OJT  it 
would  take  average  or  fast  learners  to  reach  the  same  level  of  skill  that  the  simulation-trained 
controllers  attained  within  three  weeks  following  simulation  training.  Both  baseline  and 
simulation  groups  received  approximately  one  month  of  training  prior  to  OJT. 
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TABLE  5.  STRENGTH  OF  GROUND  CONTROL  SKILLS  FOLLOWING  ONE 
MONTH  OF  SIMULATION  TRAINING  COMPARED  TO  BASELINE 


Simulation 

Baseline  (No  Simulation) 

Skill 

Strength  (1-5) 

Average  Learner: 
Strength  (1-5) 

Fast  Learner: 
Strength  (1-5) 

Visually  scans 
airport  surface 

3.21 

2.0 

2.7 

Maintains  efficient 
traffic  flow 

3.21 

2.0 

1.7 

Maintains  aircraft 
identity 

3.71 

2.0 

2.7 

Working  speed 

3.71 

1.3 

2.7 

Effectively  manages 
shipboard 

2.89 

1.7 

2.0 

Missed/delayed 
traffic  calls 

3.0 

1.0 

1.7 

Communication  is 
clear  and  concise 

3.36 

1.7 

3.0 

Makes  unnecessary 
transmissions 

2.93 

1.0 

2.0 

Overall 

3.25 

1.59 

2.31 

Table  6  shows  that  according  to  the  area  manager  and  tower  supervisors  whose  impressions  were 
obtained,  the  skills  of  developmental  controllers  who  received  simulation  training  were  advanced 
compared  to  the  skills  of  those  who  received  traditional  training.  Early  in  OJT,  simulation- 
trained  developmental  controllers  demonstrated  ground  control  skills  considered  equivalent  to 
traditionally  trained  average  learners  following  five  months  of  OJT. 

Simulation  training  was  particularly  effective  in  training  working  speed,  where  the  mean  rating  of 
3.71  indicated  that  the  trainee  could  work  at  a  speed  that  was  more  than  adequate  for  moderate 
workload  (3.0)  and  close  to  adequate  for  high  workload  (4.0).  This  finding  is  consistent  with  the 
working  speed  hypothesis  proposed  in  the  preceding  section.  This  explanation  suggested  that 
simulation  training  increased  working  speed,  enabled  the  trainee  to  handle  higher  traffic 
densities,  and  hence  permitted  the  trainee  to  work  traffic  more  hours  per  day. 
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TABLE  6.  EQUIVALENT  MONTHS  OF  OJT:  SIMULATION  TRAINING  COMPARED 

TO  BASELINE 


Simulation 

Baseline  (No  Simulation) 

Skill 

Months  of  OJT 

Average  Learner: 
Months  of  OJT 

Fast  Learner: 
Months  of  OJT 

Visually  scans 
airport  surface 

<1 

4.42 

1.51 

Maintains  efficient 
traffic  flow 

<1 

5.02 

3.7 

Maintains  aircraft 
identity 

<1 

5.42 

2.07 

Working  speed 

<1 

6.67 

4.02 

Effectively  manages 
stripboard 

<1 

4.63 

2.38 

Missed/delayed 
traffic  calls 

<1 

5.0 

3.0 

Communication  is 
clear  and  concise 

<1 

4.17 

2.17  . 

Makes  unnecessary 
transmissions 

<1 

4.9 

2.46 

Overall 

<1 

5.03 

2.66 

5.2  SIMULATION  TRAINING  MEASURES 

Each  developmental  controller  was  videotaped  once  each  week  during  simulation  training.  The 
following  objective  data  were  then  reduced  from  the  videotapes:  taxi  delay,  taxi  time,  and  the 
percentages  of  time  a  trainee’s  scan  was  directed  toward  the  “windows”  or  toward  the  simulation 
stripboard. 

Taxi  delay  and  taxi  time  provide  measures  of  general  planning  ability  and  efficiency.  Training 
was  expected  to  result  in  reduced  taxi  delay  and  taxi  time.  Stripboard  scan  time  is  a  measure  of 
the  attention  required  to  stay  organized  by  marking  and  moving  flight  strips.  Controllers  are 
trained  to  minimize  stripboard  scanning  because  it  takes  time  from  scanning  the  situation  on  the 
airport  movement  areas.  Thus  training  was  expected  to  increase  window  scanning  and  to 
decrease  stripboard  scanning. 

5.2.1  Preliminary  Data  Considerations  and  Analysis 

The  Group  3  training  data  were  excluded  from  analysis  for  several  reasons.  In  Group  3,  only  two 
developmental  controllers  of  the  three  who  began  simulation  training  completed  it:  one  needed 
to  postpone  further  training  for  medical  reasons  after  three  weeks.  The  use  of  training-to-a- 
criterion  instead  of  a  previously  set  number  of  weeks  of  training  (see  section  3.2)  resulted  in 
different  amounts  of  training  given  to  the  remaining  two  trainees  who  completed  simulation 
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training.  In  view  of  these  and  other  differences  between  Group  3  and  the  other  two  groups,  it 
appeared  inappropriate  to  combine  the  three  groups’  training  data.  Group  3  provided  insufficient 
data  (a  total  of  eight  sessions  for  three  trainees)  to  conduct  a  meaningful  separate  analysis  of 
Group  3  alone. 

The  Group  1  data  required  preliminary  analysis  to  assess  any  impact  of  the  automated  speech 
recognition/synthetic  speech  system  on  the  measures  of  training.  The  automated  system  was 
used  during  the  first  two  to  three  weeks  of  Group  1  training  (see  section  2.2.3).  It  appeared  to 
lengthen  taxi  delays  when  it  was  operating  due  to  the  need  for  the  trainees  to  repeat  taxi 
instructions  and  watch  to  see  if  indeed  the  aircraft  began  to  move  as  instructed.  By  requiring 
unusually  lengthy  attention  to  individual  aircraft  it  also  was  likely  to  have  interfered  with  the 
scan  measures. 

An  effort  was  made  to  quantify  the  effect  of  using  the  speech  recognition  system  even  though  it 
was  necessary  to  compare  across  trainee  groups.  Figure  3  shows  the  mean  taxi  delay  for  Group 
1,  during  the  first  two  weeks  of  automated  system  use  (its  use  was  discontinued  during  the  third 
week,  resulting  in  its  use  by  some  but  not  all  trainees  that  week).  The  figure  also  shows  taxi 
delay  for  the  first  two  weeks  of  Group  2  and  Group  3  training,  combined  for  comparison  to 
Group  1 .  The  Week  1  scenario  included  30  planes  for  Group  1 ,  in  contrast  to  90  planes  for 
Group  2  and  Group  3,  so  the  apparent  advantage  for  the  automated  system  is  most  likely  an 
artifact.  The  Week  2  scenarios  were  more  comparable:  Group  1  was  given  80  planes,  nearly  as 
many  as  the  90  planes  given  to  Group  2  and  Group  3.  Using  the  Week  2  data,  comparison  of  the 
first  group  while  using  the  automated  system  with  the  second  and  third  groups  suggests  that  the 
automated  system  added  an  average  of  1.77  minutes  to  the  Group  1  taxi  delay,  roughly  doubling 
this  delay.  Since  the  automated  system  appears  to  have  adversely  affected  the  Group  1  training 
data  for  the  first  two  or  three  weeks,  the  Group  1  data  was  not  analyzed  further. 
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Due  to  the  preceding  considerations,  the  only  training  data  analyzed  came  from  the  videotaped 
sessions  of  the  second  group  of  trainees.  The  four  trainees  in  Group  2  each  received  simulation 
training  for  five  weeks.  One  session  per  trainee  per  week  was  videotaped.  Each  session  lasted 
approximately  50  minutes.  Since  the  taxi  delay,  taxi  time,  and  scan  measures  were  all  based  on 
many  individual  observations,  these  data  were  considered  to  provide  a  sufficient  basis  for 
analysis. 

A  quantitative  analysis  of  instructor  comments  on  these  20  tapes  supplemented  the  training  data. 
This  analysis  was  conducted  to  provide  additional  indications  of  the  development  of  particular 
ground  control  skills  and  a  better  understanding  of  the  simulation  training  process. 

5.2.2  Taxi  Delay 

The  time  it  takes  the  ground  controller  to  provide  taxi  instructions  to  an  aircraft  that  is  ready  to 
taxi  is  its  taxi  delay.  At  O’ Hare,  most  aircraft  receive  taxi  instructions  in  less  than  30  seconds, 
which  is  considered  a  “good”  taxi  delay  time  at  this  facility^.  Taxi  delay  provides  a  general 
indication  of  a  controller’s  ability,  including  the  ability  to  identify  the  gates  and/or  alleys  used  by 
air  carriers  at  O’Hare,  communicate  efficiently,  plan  efficient  taxi  routes,  and  prioritize  actions. 

In  the  simulation,  taxi  delay  was  defined  as  the  time  interval  that  began  when  an  aircraft  was 
metered  (i.e.,  when  the  ground  meterer  placed  its  flight  strip  on  the  strip  bay)  and  ended  when  the 
aircraft  received  taxi  instructions.  It  did  not  apply  to  an  aircraft  that  was  handed  off  by  one 
ground  controller  to  another  ground  controller  (one  aircraft  in  the  videotaped  scenarios). 

Figure  4  shows  the  mean  Group  2  taxi  delay,  by  week  of  training.  It  demonstrates  that  following 
an  initial  rise,  mean  taxi  delay  decreased  monotonically  from  a  high  during  Week  2  of  almost  2.5 
minutes  to  a  low  during  Week  5  of  under  one  minute.  This  considerable  decrease  in  mean  taxi 
delay  was  accompanied  by  a  corresponding  increase  in  the  percentage  of  taxi  delays  provided  in 
at  most  30  seeonds  (Figure  5). 


FIGURE  4.  A’VERAGE  TAXI  DELAY 


7.  Matt  Dunne.  FAA  O’Hare  Tower.  Personal  Communication. 
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5.2.3  Taxi  Time 

Taxi  time  is  a  rough  estimate  of  the  time  an  aircraft  spends  taxiing  and  so  it  provides  an 
indication  of  fuel  cost  to  the  user  as  well  as  an  indication  of  how  well  a  controller  plans  taxi 
routes.  Taxi  time  varies  with  the  distance  from  the  departure  gate  to  the  point  of  handoff  (usually 
near  the  departure  runway),  and  hence  with  the  type  and  company  of  the  aircraft.  Because  it  also 
varies  with  the  distance  to  the  appropriate,  active  departure  runway,  taxi  time  also  varies  with 
airport  configuration.  Following  are  typical  taxi  times  for  O’ Hare*  when  the  airport  is  either  in 
its  “Plan  B”  or  its  “Plan  Weird”  configuration  (note  that  taxi  time  is  reduced  in  the  simulation 
due  to  the  absence  of  inbound  aircraft): 


Departure  Runway 

Taxi  Time  (Minutes) 

United 

Eastbound 

22L 

8.5 

Westbound 

32L 

3  to  4 

American 

Eastbound 

22L 

6 

Westbound 

32L 

8 

General  Aviation 

Eastbound 

22L 

<1 

Westbound 

27L 

9  to  10 

8.  Matt  Dunne.  FAA  O’Hare  Tower.  Personal  Communication. 
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As  measured  in  the  simulation  videotapes,  an  aircraft’s  taxi  time  began  when  it  was  given  taxi 
instructions  and  ended  when  it  was  handed  off  to  the  local  controller.  The  airport  configuration 
or  Plan  was  held  constant  (Plan  Weird)  in  the  Group  2  sessions  videotaped. 

Figure  6  shows  the  mean  taxi  times  by  week  of  training  for  Group  2.  It  is  evident  that  taxi  time 
remained  relatively  stable  during  the  five  weeks  of  training.  The  taxi  times  measured  were 
considerably  lower  than  most  of  those  considered  typical  at  O’ Hare.  It  is  likely  that  the  times 
were  low  in  part  because  no  inbound  aircraft  were  simulated.  It  is  not  clear  why  taxi  time  did  not 
reflect  the  benefit  of  simulation  training;  however,  it  is  possible  that 

-  taxi  time  depends  mainly  on  the  distance  from  the  departure  gate  to  the  appropriate 
runway,  with  little  effect  of  controller  skill. 

-  the  ground  controller  has  more  flexibility  in  how  aircraft  are  handed  off  to  the  local 
controller,  the  event  which  closed  taxi  time  in  this  study.  Depending  upon  the  situation,  a 
handoff  could  occur  in  the  same  message  as  a  taxi  instruction,  or  it  could  wait  until  a  later  time 
when  the  trainee  had  an  opportunity  to  make  several  handoffs  in  a  single  transmission.  Time-to- 
runway  (if  measured)  might  show  better  sensitivity  to  training  effects  than  time-to-handoff. 

-  the  longer  taxi  time  durations,  compared  to  the  length  of  taxi  delays,  made  them  more 
likely  to  include  the  time  taken  by  instructors’  assistance  and  comments. 

-  increases  in  skill  were  counteracted  by  increases  in  training  program  demands.  Some  of 
the  later  training  demands  such  as  traffic  calls  and  splitting  fixes  could  affect  taxi  time  more  than 
taxi  delay. 
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5.2.4  Window  and  Stripboard  Scan 


This  pair  of  measures  describes  the  length  of  time  each  trainee’s  visual  scan  or  gaze  was  directed 
toward  the  simulation’s  visual  displays  (i.e.,  toward  the  simulated  tower  windows),  the 
stripboard,  or  elsewhere  in  the  training  room  (typically  toward  an  instructor).  Window  scanning 
and  stripboard  scanning  were  of  particular  interest  because  of  anecdotal  reports  that  some 
trainees  spend  too  much  time  looking  at  flight  strips  and  consequently  insufficient  time  looking 
through  tower  windows  at  the  aircraft  movement  areas. 

Stripboard  management  began  to  require  less  visual  attention  after  the  second  week  of  training. 

It  decreased  between  the  second  and  fifth  weeks  from  51.4%  to  44.8%  of  the  total  time.  Window 
scanning  began  to  increase  after  the  third  week  from  44.0%  to  50.2%  of  the  total  time.  The 
direction  of  these  moderate  changes  in  gaze  is  consistent  with  the  hypothesis  that  simulation 
training  improves  stripboard  management  skills.  This  evidence  is  presented  in  Figure  7. 


FIGURE?.  SCAN  TOWARD  WINDOW 
AND  STRIPBOARD 


5.2.5  Analysis  of  Instructor  Assistance 

The  constraints  under  which  this  study  was  conducted  (see  section  1 .3)  require  a  research 
strategy  in  which  different  types  of  evidence  are  examined  for  consistent  trends.  Accordingly, 
videotaped  instances  of  instructor  assistance  were  analyzed  to  provide  converging  evidence  on 
the  development  of  specific  ground  control  skills. 

Instructors  made  782  discernible  suggestions  and  comments  during  the  20  Group  2  training 
sessions  that  were  videotaped  and  used  in  this  evaluation.  This  instructor  advice  was  examined 
to  determine  whether  the  trainees  required  less  advice  as  training  progressed.  Each  instructor 
comment  was  classified  according  to  topic  to  provide  additional  indications  of  the  development 
of  particular  ground  control  skills. 
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The  instructor  comments  were  classified  into  twelve  categories.  Six  were  designated  as  “major 
categories”  and  six  were  designated  as  “minor”  on  the  basis  of  frequency.  The  major  categories 
included  90.15%  of  the  comments.  For  each  week  of  training,  the  mean  amount  of  advice 
(number  of  comments)  in  each  of  the  major  and  minor  categories  is  shown  in  Table  7.  Figure  8 
shows  the  effect  of  training  on  the  average  amount  of  assistance  (per  major  category)  trainees 
required  to  perform  the  problem.  A  trainee  required  roughly  half  as  much  assistance  by  the  end 
of  simulation  training. 

Table  7  shows  reductions  in  advice  on  stripboard  management  and  taxi  routes,  which  appear  to 
have  reached  asymptotes  (minimum  levels)  by  the  end  of  simulation  training.  Priority  of  duty, 
(i.e.,  knowing  which  aircraft  to  call  next)  required  instmctor  comments  primarily  during  the  first 
two  sessions,  after  which  comments  appeared  at  a  much  reduced  rate.  Situational  awareness  is 
the  only  major  topic  that  appears  to  have  required  a  fairly  constant  level  of  instructor  attention 
throughout  training. 


TABLE  7.  INSTRUCTOR  ASSISTANCE  BY  WEEK  OF  TRAINING  (MEAN  NUMBER 

OF  ASSISTS) 


Categories  Of  Assistance 

Weeks  of  Training 

Major  Categories 

1 

2 

3 

5 

Stripboard  Management 

10 

3.25 

Situational  Awareness 

5.25 

4.5 

4.75 

Priority  of  Duty 

10 

9 

3.25 

3 

2 

Communications 

9.75 

7.5 

6 

Taxi  Routes 

9.75 

9 

5.5 

5.5 

Traffic  Calls 

4.75 

3.5 

3.5 

2.75 

1 

Minor  Categories 

1 

2 

3 

4 

5 

Splitting  Fixes 

.5 

3 

ira 

1.75 

Timely  Taxi  Instructions 

.5 

1 

.25 

.75 

Handoffs  to  Local  Controller 

.75 

.25 

0 

0 

Aircraft  Identification 

1 

.25 

Sequencing  Aircraft 

.75 

.25 

1.25 

.5 

.5 

Runway  Assignment 

.5 

.25 

.5 

0 
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As  training  progressed,  instructors  made  fewer  comments  on  the  developmental  controllers’ 
communications.  The  highest  number  of  comments  were  made  during  the  first  and  second 
sessions.  Compared  to  an  average  of  approximately  ten  comments  made  in  those  initial  sessions, 
the  final  session  shows  an  approximate  40%  reduction.  Over  the  same  period,  the  trainees 
increased  the  number  of  communication  errors  they  caught  and  corrected  themselves  by  more 
than  60%.  Figure  9  presents  these  results. 


FIGURE  9.  INSTRUCTOR-CORRECTED 


1  2  3  4  5 

Weeks  of  Training 


28 


6.  FINDINGS  AND  RECOMMENDATIONS 


This  section  summarizes  and  draws  conclusions  from  the  evidence  presented  in  the  preceding 
section.  This  evidence  was  gathered  during  the  first  year  of  training  using  a  prototype  out-of-the- 
window  tower  training  simulator  at  O’Hare  International  Airport  in  Chicago.  The  simulator  was 
used  to  train  the  skills  and  knowledge  required  to  perform  outbound  ground  control  at  O’ Hare. 
The  evidence  supports  the  following  conclusions: 

The  out-of-the-window  tower  simulation  used  at  O’Hare: 

•  is  an  effective  tool  for  training  many  ground  control  skills  and  knowledge;  it  appears 
especially  effective  in  increasing  trainees’  working  speed. 

•  is  more  effective  for  training  ground  control  than  more  traditional  training  techniques. 

6.1  TRAINING  USING  THE  SIMULATOR  WAS  EFFECTIVE 

During  training,  the  ground  control  performance  of  the  developmental  controllers  studied 
consistently  improved,  beginning  with  the  second  week  of  training.  Following  a  month  of 
simulation  training,  their  ground  control  performance  (in  the  actual  tower)  was  rated  as  better 
than  the  performance  of  traditionally  trained  developmental  controllers  (see  section  6.2).  The 
following  findings  indicate  improvements  during  the  simulation  training  sessions,  but  do  not 
permit  comparisons  between  simulation  and  traditional  training.  The  findings  from  comparisons 
between  simulation  and  non-simulation  training  (listed  in  section  6.2)  suggest  that  the 
improvements  in  ground  control  skills  described  in  this  section  were  due  either  to  simulation 
training  alone  or  to  a  combination  of  simulation  and  concurrent  non-simulation  training. 

Finding:  Taxi  delay  decreased  consistently  from  the  second  week  to  the  end  of  training. 

The  evaluation  found  a  50%  decrease  in  average  taxi  delay  duration,  with  a  tripling  of  the 
percentage  of  taxi  delays  under  30  seconds  in  a  comparison  of  the  second  and  fifth  (last)  weeks 
of  training.  The  monotonic  downward  trend  was  evident  during  each  intervening  week. 

Finding:  Stripboard  scanning  began  to  consistently  decrease  after  the  second  week  of  training, 
while  window  scanning  began  to  increase  after  the  third  week  of  training. 

Moderate  improvements  in  these  two  objective  measures  were  found.  Stripboard  management 
began  to  require  less  visual  attention  after  the  second  week  of  training,  and  ultimately  decreased 
from  51.4%  to  44.8%  of  the  total  scan  time,  a  7.7%  decrease.  Window  scanning  increased  after 
the  third  session,  from  44.0%  to  50.2%,  a  6.2%  increase. 

Finding:  The  amount  of  assistance  needed  for  a  trainee  to  properly  handle  a  scenario  decreased 
consistently  throughout  training. 
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The  evaluation  found  a  decrease  of  approximately  50%  in  the  number  of  instructor  assists  from 
the  first  to  the  last  week  of  training.  The  number  of  assists  decreased  in  every  major  category  of 
ground  control  skill.  The  major  categories  included  Stripboard  Management,  Situational 
Awareness,  Priority  of  Duty,  Communications,  Taxi  Routes,  and  Traffic  Calls.  Stripboard 
Management  and  Priority  of  Duty  showed  the  largest  decreases. 

6.2  TRAINING  USING  THE  SIMULATOR  WAS  FASTER  AND  MORE  EFFECTIVE  THAN 
TRADITIONAL  TRAINING 

Developmental  controllers  became  certified  on  the  outbound  ground  control  position  in  fewer 
days  following  a  month  of  simulation  training.  Supervisors’  ratings  of  their  ground  control  skills 
indicated  that  they  were  further  advanced,  compared  with  the  results  of  traditional  training. 

Finding:  The  developmental  controllers  who  were  trained  using  simulation  became  certified  on 
the  outbound  ground  control  position  in  fewer  days,  but  required  about  the  same  total  number  of 
hours,  as  developmental  controllers  who  were  trained  without  simulation. 

Using  facility  records,  comparisons  were  made  on  the  number  of  days  and  the  number  of.hours  of 
OJT  taken  to  become  certified  on  the  outbound  ground  control  position.  The  evaluation  found 
that  it  took  the  simulation-trained  developmental  controllers  an  average  (mean)  of  149  days  to 
become  certified  on  the  outbound  ground  control  position.  In  contrast,  it  took  the  developmental 
controllers  who  became  certified  on  outbound  ground  control  prior  to  simulation  training  199 
days,  a  statistically  significant  difference  of  approximately  25%.  The  two  groups  involved  in 
these  comparisons  had’  received  approximately  the  same  amount  of  training  prior  to  OJT. 

The  difference  in  days-to-certification  found  between  developmental  controllers  trained  using 
simulation  and  those  trained  without  simulation  did  not  appear  to  extend  to  total  hours-to- 
certification.  The  evaluation  found  that  simulation-trained  developmental  controllers  took  only 
5%  less  time  in  terms  of  total  hours-to-certification.  This  pair  of  results  provides  part  of  the 
evidence  favoring  a  working  speed  hypothesis. 

The  Working  Speed  Hypothesis: _ _ _ _ _ 

Simulation  training  increased  developmental  controllers’  working  speed,  which  enabled  them  to 
work  in  the  actual  tower  under  a  wider  range  of  conditions  (i.e.,  under  heavier  or  more  complex 
traffic)  and  hence  for  more  hours  per  day  than  with  traditional  training.  Thus,  simulation 
training  decreased  the  number  of  days  but  not  the  total  number  of  hours  needed  for  certification 
on  position. _ _ _ _ _ _ 


Additional  evidence  in  support  of  the  working  speed  hypothesis  is  found  in  the  supervisor  ratings 
of  eight  skill  categories,  where  working  speed  was  rated  highest  compared  to  a  baseline  of 
traditional  training.  It  received  a  mean  rating  of  3.71 ,  which  indicates  that  the  trainee  could  work 
at  a  speed  that  is  more  than  adequate  for  moderate  workload  (3.0)  and  close  to  adequate  for  high 
workload  (4.0).  Baseline  ratings  for  controllers  given  traditional  training  were  1 .3  for  average 
learners  and  2.7  for  fast  learners.  The  working  speed  hypothesis  is  also  consistent  with  the 
finding  of  a  50%  drop  in  taxi  delay  during  the  course  of  simulation  training. 
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That  simulation  training  did  not  appear  to  affect  hours-to-certification  suggests  that  simulation 
training  did  not  address  at  least  some  critical  ground  control  skills  (see  the  following  section  6.3). 
These  skills  apparently  develop  independent  of  working  speed  so  that  an  increased  working 
speed  does  not  reduce  the  number  of  hours  of  OJT  needed  to  develop  these  other  skills. 

Finding:  Supervisor  ratings  made  after  the  completion  of  simulation  training  were  higher  than 
the  ratings  of  traditionally  trained  developmental  controllers. 

In  general,  the  ratings  made  by  tower  supervisors  indicated  that  within  three  weeks  following 
simulation  training,  developmental  controllers  exhibited  ground  control  skills  in  the  actual  tower 
comparable  to  a  traditionally  trained  “average  learner”  after  five  months  of  OJT  or  to  a  “fast 
learner”  after  2.7  months  of  OJT.  The  supervisors  rated  all  eight  categories  of  ground  control 
skill  higher  following  simulation  training  than  following  traditional  training,  even  when 
compared  to  fast  learners.  Simulation  training  appeared  most  effective  in  increasing  working 
speed. 

6.3  TRAINING  USING  TOWER  SIMULATION  IS  LIKELY  TO  SHOW  INCREASED 
BENEFITS  UPON  UPGRADING 

One  difficulty  of  traditional  OJT  for  outbound  ground  controllers  is  that,  at  times,  the  volume  of 
traffic  and  its  complexity  can  become  so  high  that  the  trainee  cannot  proceed  without  potentially 
compromising  the  safe  and  efficient  flow  of  aircraft  to  their  runways.  At  these  times,  the  trainee 
must  leave  his  or  her  post  and  allow  the  instructor  to  take  over.  This  reduces  the  amount  of  time 
for  OJT  on  that  day.  The  decreased  taxi  delays  and  higher  supervisor  ratings  found  in  this  study 
are  consistent  with  the  notion  that  simulation  training  increased  the  working  speed  of 
developmental  controllers  to  the  point  where  they  could  benefit  from  OJT  even  under  conditions 
of  moderate-to-high  density  and  complexity. 

The  25%  difference  in  days-to-certification  following  simulation  training  was  not  accompanied 
by  a  commensurate  reduction  in  hours-to-certification.  These  results  suggest  that  some  crucial 
ground  control  skills  are  not  learned  in  simulation  training  as  currently  implemented.  To  the 
extent  that  these  skills  can  be  identified  and  simulation  training  enhanced,  extended^,  or 
upgraded  to  support  teaching  them,  one  can  expect  additional  benefits.  Recommendations  in  this 
section  are  offered  under  the  assumption  that  the  FAA  decides  to  proceed  with  tower  controller 
training  simulation. 


9.  Many  of  the  recommended  simulation  enhancements  and  new  application  areas  (section  6.4)  involve  extending 
simulation  training.  This  would  increase  training  staff  time  and  costs.  The  simulation’s  speech  recognition  and 
synthetic  speech  capability  was  intended  to  enable  a  trainee  to  practice  without  a  simulation  operator  continually 
present.  When  the  simulation  speech  system  was  operating,  aircraft  would  automatically  follow  their  assigned 
routes,  and  pilots  would  read  back  instructions  and  clearances  automatically,  using  synthetic  voice.  Problems 
encountered  in  the  use  of  the  evaluated  simulation’s  speech  recognition  and  synthetic  speech  systems  are  described 
in  section  2.2.3.  If  these  problems  are  solved,  autonomous  capabilities  could  reduce  the  scheduling  and  financial 
costs  associated  with  some  of  these  enhancements  and  applications.  Autonomous  simulation  capabilities  could  also 
increase  the  accuracy  of  assessments  made  in  a  simulated  tower  environment  by  increasing  control  over  test 
conditions. 
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The  evidence  obtained  in  this  evaluation  suggests  that  upgrades  are  required  to  train  or  to 
increase  the  effectiveness  of  training  on  the  following  skills: 

1 .  Inbound  ground  control  and  coordination  between  inbound  and  outbound  ground  control 
positions 

2.  Reflexive,  correct  communications 

3.  Scanning  and  situational  awareness 

4.  Understanding  what  a  pilot  can  see  from  the  cockpit 

5.  Smooth  transition  between  window  scan  and  BRITE/ASDE  displays 

6.  Teamwork 

Inbound  ground  control  and  coordination  between  inbound  and  outbound  ground  control 
positions. 

Without  inbound  aircraft,  the  simulation  could  not  provide  the  complexity  of  the  taxi  patterns  at 
O’Hare.  This  limitation  reduced  the  maximum  difficulty  of  the  training  problems,  and  it  thus 
reduced  the  number  of  weeks  the  simulation  could  present  challenging  problems  for  training.  All 

of  the  upgraded  capabilities  listed  below  that  involve  extending  the  duration  of  simulation 
training  require  a  capability  for  inbound  aircraft. 

Ground  control  training  conducted  in  a  simulation  lacking  inbound  aircraft  could  have  led  to  the 
development  of  unrealistic^expectations  of  the  pace,  decisions,  and  actions  needed  to  move  from 
one  point  to  another  because  the  outbound  aircraft  did  not  have  to  stop  at  the  runways  and 
taxiways  used  for  arrivals.  Also,  without  inbound  aircraft,  trainees  did  not  have  an  opportunity 
to  learn  the  coordination  skills  needed  to  work  outbound  ground  control  in  tandem  with  an 
inbound  ground  controller. 

Recommendations: 

•  Upgrade  simulation  technology  to  support  inbound  aircraft 

•  Upgrade  simulation  technology  to  include  an  additional  position  to  be  used  for  inbound 
ground  control 

•  Extend  the  duration  of  simulation  training. 

Reflexive,  correct  communications.  It  appears  that  extended  simulation  training  could  improve 
trainee  communication  skills.  As  the  emphasis  during  the  month  of  simulation  training  changed 
from  fundamental  to  advanced  skills,  so  did  the  content  of  controller  communications  and  their 
associated  phraseology.  This  continual  change  in  what  was  required  of  trainee  communications 
could  have  produced  the  finding  that  more  instructor  assistance  was  provided  in  the 
communications  category  than  in  any  other  category  during  the  last  recorded  training  session.  A 
related  finding  was  that  the  total  number  of  corrections  in  the  communications  category  (self¬ 
corrections  and  instructor  corrections)  remained  fairly  constant  throughout  training.  A  trend 
toward  less  instructor  assistance  on  communications  after  the  second  week  of  training  suggests 
that  trainees  were  learning  better  communication  skills,  but  at  end  of  simulation  training, 
communications  skills  were  among  the  least  advanced,  compared  to  traditional  training,  in 
supervisors’  ratings.  Together,  these  findings  suggest  that  additional  training  could  further 
promote  the  reflexive,  correct  communications  that  controllers  require  at  airports  like  O’Hare, 
which  are  characterized  by  high  aircraft  density  and  complex  traffic  patterns. 
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Recommendation: 

•  Extend  the  duration  of  simulation  training. 

Scanning  and  situational  awareness.  Some  results  indicated  that  technical  and/or  procedural 
aspects  of  simulation  training  could  be  enhanced  to  increase  the  amount  of  time  developmental 
controllers  scan  the  airport.  Objectively,  scanning  out  the  simulation  windows  increased  6.2% 
during  the  last  two  weeks  of  training.  During  this  time,  the  amount  of  instructor  assistance  on 
situational  awareness  decreased,  but  it  remained  almost  as  high  as  at  the  start  of  simulation 
training.  In  supervisor  ratings  made  soon  after  the  completion  of  simulation  training,  scanning 
the  airport  was  a  skill  rated  less  than  average  in  strength,  and  less  advanced  compared  to  the 
other  skills  rated.  These  results  suggest  that  scanning  and  situational  awareness  began  to 
improve  relatively  late,  and  could  benefit  from  additional  simulation  training.  Possibly,  the  need 
to  pan  the  displayed  view  of  the  airport  may  have  slowed  the  integration  of  scanning  into  larger 
units  of  ground  control  skill,  and  may  have  limited  the  transfer  of  scanning  skills  from  the 
simulation  to  the  actual  tower. 

Recommendations 

•  Enhance  the  simulation  to  simultaneously  display  the  entire  airport. 

•  Extend  the  duration  of  simulation  training. 

Understanding  what  a  pilot  can  see  from  the  cockpit.  Situational  awareness  for  ground  and  local 
controllers  includes  an  awareness  of  what  pilots  are  likely  to  see  at  any  given  moment.  This 
awareness  is  important  when  communicating  instructions  which  commonly  refer  to  aircraft 
relative  to  the  pilot’s  position,  as  in  “Join  alfa  behind  the  MD80  off  your  left.”  Situations  arose 
during  simulation  training  which  suggested  that  ground  controllers  need  training  on  what  pilots 
are  likely  to  see.  For  example,  an  instructor  asked,  “Which  company  do  you  think  he’s  looking 
at?”  On  another  occasion,  an  instructor  commented  on  a  trainee’s  instruction:  “He  may  never  see 
American  off  his  left.”  A  simulation  capable  of  showing  a  realistic  pilot’s  eye  view  could 
advance  training  on  this  aspect  of  situational  awareness. 

Recommendation 

Provide  a  “pilot’s  eye  view”  capability  that  will  realistically  represent  the  view  of  a  pilot  from  the 
cockpits  of  particular  types  of  aircraft. 

Smooth  transition  between  window  scan  and  BRITEIASDE  displays. 

Developmental  controllers  could  use  current  tower  simulation  to  learn  to  translate  aircraft 
locations  as  displayed  on  BRITE  and  ASDE  systems  from/to  their  locations  as  seen  through  the 
tower  windows.  Training  in  this  advanced  skill  could  lead  to  increased  situational  awareness. 

Recommendations 

•  Upgrade  the  simulation  technology  to  provide  an  interface  to  BRITE  and  ASDE  displays 
showing  aircraft  coordinated  to  the  simulation. 

•  Extend  the  duration  of  simulation  training. 
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Teamwork.  The  simulation  permits  training  for  only  one  tower  position.  With  enhancements, 
the  simulation  could  be  used  to  train  developmental  controllers  to  participate  in  the  coordinated 
team  effort  required  to  control  aircraft  at  O’Hare.  Teaching  teamwork  and  coordination  skills 
requires  the  simulation  to  simultaneously  show  the  entire  airport.  This  capability  would  enable 
two  or  more  controllers  to  view  different  parts  of  the  airport  at  the  same  time. 

Recommendations 

•  Upgrade  simulation  technology  to  support  inbound  aircraft. 

•  Upgrade  simulation  technology  to  include  an  additional  position  for  inbound  ground  control 
or  local  control. 

•  Enhance  the  simulation  to  simultaneously  display  the  entire  airport. 

•  Extend  the  duration  of  simulation  training. 

6.4  CURRENT  COST  OF  RECOMMENDED  UPGRADING 

The  recommended  upgrades  are  all  currently  available  from  the  U.S.  manufacturer  of  the 
prototype,  Wesson  International  (Aviation  Simulatons,  Inc.  is  a  joint  venture  of  Wesson 
International,  Inc.  and  the  British  Bruce  Artwick  Organization,  Ltd.).  Costs  of  the  personal 
computer-based  system  depend  primarily  upon  the  number  of  controller  or  pseudo 
pilot/simulation  operator  stations,  and  on  the  number  of  projection  screen  displays.  All  current 
systems  include  simulated  BRITE  and  ASDE  displays  and  “pilot’s  eye  view”  capabilities.  A 
system  that  includes  all  recommended  upgrades  (eight  displays  for  a  360  degree  view  and  three 
positions  -  two  controller  stations  and  a  pseudo  pilot/simulation  operator  station)  would  currently 
cost  $1,050,950.  A  new  system  representative  of  the  one  evaluated  at  O’Hare  (three  displays  for 
a  135  degree  view,  and  one  controller  position,  and  a  pseudo  pilot/simulation  operator  station) 
would  currently  cost  $495,950. 

6.5  NEW  APPLICATIONS  OF  TOWER  SIMULATION  TECHNOLOGY 

Simulation  training  requires  use  of  the  simulator  for  roughly  six  months  per  year.  Out-of-the- 
window  ATC  tower  simulators  offer  additional  potential  uses  which  could  be  scheduled  during 
the  rest  of  the  year,  when  the  simulator  is  not  required  for  training.  The  first  topics  concern 
advanced  training  for  current  full  performance  level  (FPL)  controllers.  Sections  follow  on  the 
use  of  tower  simulation  for  candidate  screening,  and  for  the  assessment  of  new  tower  training 
methods  and  technology  and  the  evaluation  of  new  automation  systems,  procedures,  and  airport 
configurations. 

Individual  performance  enhancement  for  current  controllers.  The  simulation  evaluated  does  not 
appear  able  to  provide  sufficient  volume  and  complexity  to  increase  the  skills  of  current  FPL 
controllers  at  O’Hare  or  at  other  high  density  or  high  complexity  airports.  It  would  be  essential 
to  add  a  large  volume  of  inbound  aircraft  to  increase  the  difficulty  of  the  training  scenarios  if 
training  is  to  benefit  current  FPL  controllers.  This  training  could  possibly  enable  them  to  provide 
better  services  and  safety  and  to  develop  strategies  for  handling  more  traffic. 
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Team  performance  enhancement  for  current  controllers.  The  current  one-position  simulation 
does  not  provide  opportunities  to  train  tower  controllers  to  work  more  effectively  together  as  a 
team.  A  multiple-position  simulation  with  sufficient  volumes  of  inbound  and  outbound  aircraft 
would  permit  practice  and  training  on  situations  requiring  coordination  and  teamwork.  The 
objectives  of  this  training  would  be  similar  to  those  of  crew  resource  management  or  CRM^®. 

For  example,  they  might  include  improvements  in  end-of-shift  briefings,  workload  distribution, 
situational  awareness,  management  of  abnormal  situations,  and  error  surveillance. 

Training  in  the  handling  of  unusual  situations.  Using  simulation,  it  would  be  possible  to  train 
tower  controllers  to  recognize  and  properly  handle  situations  rarely  or  infrequently  seen  in  actual 
operations.  Without  operational  experience,  simulation  would  appear  to  offer  the  best  way  for 
controllers  to  learn  to  contend  with  the  overall  consequences  of  such  situations  for  airport  traffic 
control.  Such  situations  include  emergency  landings  requiring  passenger  evacuation,  hijacked 
aircraft  or  other  aviation  security  events,  landings  of  diplomatic  aircraft,  aircraft  taxiing  on 
runways  incapable  of  supporting  their  weight,  fuel  spills,  birds  or  other  wildlife  on  runways, 
unusual  and  dangerous  airport  weather  (e.g.,  icy  runways,  windshear  emergencies,  or  tornadoes), 
mechanical  failures  on  aircraft,  emergency  medical  flights,  and  aircraft  with  similar  callsigns 
approaching  a  runway.  Procedures  for  many  of  these  situations  appear  amenable  to  simulation 
training  and  practice. 

The  optimal  conditions  for  training.  Out-of-the-window  tower  simulation  technology  offers  the 
ability  to  control  the  training  environment  to  a  greater  extent  than  the  operational  environment 
permits.  The  training  staff  at  O’ Hare  took  advantage  of  this  new  capability  during  the  first  year 
of  simulation  training  by  tailoring  the  volume  of  aircraft  and  the  amount  of  guidance  presented  to 
the  individual,  and  by  modifying  their  training  techniques  between  groups  of  trainees  (see  section 
3.3). 

This  ability  to  control  the  training  environment  could  lead  to  further  efforts  toward  optimizing 
tower  controller  training.  Training  specialists  could  apply  laboratory  findings  on  the  effects  of 
experimental  training  conditions.  For  instance,  in  one  study  (Vidulich,  Yeh,  and  Schneider, 
1983^^),  a  group  of  subjects  that  practiced  a  simulated  air  traffic  control  task  that  ran  at  twenty 
times  real  time  performed  better  in  some  aspects  (and  the  same  in  all  other  areas)  than  a  group 
that  practiced  at  real  time.  Algorithms  for  time-compressed  information  presentation  have 
recently  been  developed  .  The  possibility  of  applying  the  lessons  of  such  developments  clearly 
depends  upon  the  availability  of  adequate  tower  simulation  facilities. 

Tower  controller  candidate  screening.  Training  resources  are  best  allocated  to  the  most  highly 
qualified  candidates.  Current  selection  procedures  consider  only  how  a  candidate  appears  on 
paper.  The  tower  personnel  who  must  select  among  candidates  for  a  limited  number  of  training 


10.  Taggart,  W.R  (1994).  Crew  resource  management:  Achieving  enhanced  flight  operations.  In  N.  Johnson,  N. 
McDonald,  and  R.  Fuller  (eds).  Aviation  Psychology  in  Practice,  Aldershot,  Hants,  U.K.,  Averbury. 

11.  Vidulich,  M.,  Yeh,  Y.,  and  Schneider,  W.  (1983).  Time-compressed  components  for  air-intercept  control  skills. 
Proceedings  of  the  27th  Meeting  of  the  Human  Factors  Society. 

12.  Guckenberger,  D.,  Guckenberger,  L.  Luongo,  F.,  Stanney,  K.,  and  Sepulveda,  J.  (April,  1995).  Above-real-time 
training  and  the  hyper-time  algorithm.  Dr.  Dobb’s  Journal,  52-61 . 
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slots  have  no  direct  knowledge  of  how  well  the  candidates  can  perform  under  realistic  task 
demands.  They  also  lack  clear  evidence  of  motivational  factors  which  can  lead  a  candidate  to 
expend  special  effort  at  preparation,  such  as  studying  the  layout  of  the  airport  and  surrounding 
airspace.  In  a  realistic  simulation,  candidates  can  demonstrate  their  current  knowledge  and  skills 
without  affecting  operations.  Ideally,  the  tower  staff  would  observe  a  candidate’s  demonstration 
of  prerequisite  knowledge  and  skills,  and  the  ability  to  benefit  from  training  in  a  simulation 
screening  process  which  extends  over  several  days.  A  draft  document  which  describes  the 
organization  of  a  training  candidate  screening  process  using  simulation  is  available  from  O’Hare 
Tower^^. 

Assessment  of  new  tower  equipment,  procedures,  and  airport  configurations:  Cost-benefit 
studies. 

New  tower  automation  systems,  new  procedures  such  as  coded  taxi  routes,  and  new  airport 
configurations  (often  involving  new  or  extended  runways)  promise  to  increase  airport  capacity 
and  to  improve  airport  safety  and  the  services  provided  by  tower  controllers.  Assessment  is 
needed  to  determine  the  extent  to  which  the  goals  of  each  new  system,  procedure  or 
configuration  are  reached,  to  determine  its  effect  on  the  overall  tower  environment,  and  to  assess 
proposed  design  modifications.  Although  the  assessment  of  each  new  system  can  include  unique 
considerations,  controllers’  visual,  manual,  pilot  communications  and  coordination  workload, 
and  their  response  times,  errors,  decision  making,  and  situational  awareness  are  common 
considerations.  Such  assessments  must  first  be  made  in  the  context  of  the  current  tower 
environment  to  provide  a  baseline  for  comparison  with  the  new  automation  system,  airport 
configuration,  or  procedure. 

Some  pertinent  measures  cannot  be  made  in  an  operational  environment  because  they  could 
interfere  with  tower  operations  or  because  the  necessary  control  over  the  situation  is  lacking. 
Examples  include  testing  controller  response  time  to  identify  the  cause  of  alarms  and  respond 
appropriately,  to  identify  and  correct  a  controller  error,  or  to  identify  and  correct  pilot  deviations 
from  controller  instructions.  In  these  cases,  the  initiating  events  occur  too  rarely  in  actual 
operations  and  would  have  to  be  simulated.  It  would  also  appear  impractical  to  test  the  actual 
consequences  of  automated  taxi  route  decisions,  or  to  determine  whether  controllers  can  safely 
monitor  them.  The  measurement  of  situational  awareness  technically  requires  stopping  the 
situation  to  collect  data*'*.  Other  measures,  such  as  the  frequency  of  manual  activity  and  data 
input  errors,  are  best  measured  automatically  through  direct  connections  to  the  device, 
connections  which  would  require  fabrication.  In  general,  the  most  accurate  and  relevant 
measurements  can  be  made  in  an  operational  situation,  in  the  presence  of  realistic  workload, 
distractions,  time  demands,  and  stress.  However,  as  these  examples  suggest,  the  operational 
situation  may  not  be  suitable  to  the  required  measures,  requiring  instead  the  use  of  global  or 
subjective  measures  which  could  lead  to  less  accurate  conclusions. 
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Out-of-the-window  tower  simulation  could  provide  a  more  convenient  and  practical  testbed  for 
the  assessment  of  new  tower  automation  systems  than  operating  towers.  It  would  appear 
particularly  useful  in  the  assessment  of  automation  systems  to  be  used  by  local  and  ground 
controllers,  whose  work  depends  upon  viewing  aircraft  movement  areas.  A  list  of  recommended 
requirements  for  an  assessment  simulator  is  presented  in  Appendix  C. 
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APPENDIX  A:  FOCUS  GROUP  SUMMARY 


This  focus  group  was  conducted  with  four  developmental  controllers  in  the  first  group  after  they 
had  7  to  10  hours  working  at  the  outbound  ground  control  position,  following  20  hours  (4  weeks)  of 
simulation  training.  Following  is  an  edited  summary  of  the  session. 

In  what  ways  is  actual  ground  control  different  from  the  simulation? 

-  Because  of  the  need  to  scroll  the  simulation  it  is  harder  to  keep  track  of  AC  in  peripheral  vision 
and  to  know  when  to  transfer  the  AC. 

-  Simulation  workload  is  constant  because  it  presents  planes  at  a  regular  pace,  unlike  the  real  tower 
where  the  workload  occurs  in  "clumps".  There  is  no  "breathing  time"  in  the  simulation. 

-  The  simulation's  graphic  resolution  is  sometimes  "grainy"  and  its  colors  less  intense,  adding  a  half 
second  to  the  time  it  takes  to  identify  aircraft.  For  example,  one  must  reason  from  type  (737)  to 
company  (not  American,  therefore  United)  to  check  yourself;  upstairs  you  would  know  instantly. 

-  More  small  planes.  Eagles  and  Gulfs,  in  actual  traffic  than  in  simulated.  Maybe  needs  updating. 

In  what  ways  was  the  simulation  an  effective  teaching  tool? 

-  Learning  taxi  routes. 

-  It  eliminates  worry  about  real  aircraft  moving  around  and  the  training  specialist  stepping  in  to  fix 
something  instantly  or  to  make  something  work.  It  provides  a  less  pressured,  more  relaxed 
environment  instead  of  needing  to  fix  it  instantly. 

-  It  permits  control  over  the  pace  of  the  problem.  Upstairs  when  its  slow  its  too  slow;  when  busy  it 
is  too  busy;  there  seems  to  be  no  happy  medium. 

Was  the  pace  of  the  simulation  optimally  challenging  throughout  all  four  weeks? 

-  It  began  a  little  too  slow  at  1  plane  per  minute,  and  too  fast  at  the  end  (1  per  30  sec)  when  learning 
to  make  splits.  The  speed  jumped  up  sometimes  before  I  caught  on. 

If  there  was  another  month  of  simulation  what  would  you  like  to  cover? 

-  Splits,  and  more  varied  scenarios,  including  surges  and  rest,  different  planes  and  different  rushes, 
instead  of  always  starting  up  west  and  ending  up  with  west. 

-Rehearsal  of  actual  rushes  and  regularly  scheduled  flights  in  all  configurations. 

Any  advice  for  the  next  group  of  developmentals  to  receive  simulation  training? 

-  Focus  on  managing  stripboard. 

-  Don't  worry  about  making  mistakes  or  looking  stupid,  and  ask  questions  instead  of  pretending  to 
understand. 

How  might  the  simulation  be  used  better? 

-  Instructors  should  use  the  pause  feature  more  to  allow  evaluation  of  what  has  just  occurred,  in 
smaller  chunks,  identifying  problem  and  solution. 

-  Speed  up  the  problems  at  the  beginning  to  make  them  more  realistic. 

-  Reduce  the  speed  of  a  problem  when  first  increasing  its  complexity. 

-  Use  1  or  2  instructors  full  time  instead  of  many  instructors  because  they  teach  different  techniques 
for  doing  the  same  thing. 
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According  to  the  trainees,  the  following  topics  are  learned  better  with  live  traffic  than  through 
simulation  training  (although  with  "additional  tweaking"  the  simulation  could  help  with  these): 

-  Split  fixes. 

-  Relative  taxi  speeds. 

-  Location:  It  is  easier  to  see  planes  waiting  for  departure  on  the  approach  end  of  a  taxiway  in  the 
real  tower,  than  on  the  simulator.  -  Visual  scan:  One  learns  to  move  around  in  the  actual  tower, 
whereas  one  always  looks  straight  ahead  in  the  simulation.  One  does  learn  where  to  scan  in  the 
simulation. 

-  Aircraft  identification:  The  simulation  picture  is  not  sharp  enough  to  identify  companies. 

Did  the  simulation  help  you  learn... 

Efficient  traffic  flow? 

-  Yes.  It  provides  practice  on  maintaining  continuous  attention  and  not  wasting  time 
Making  traffic  calls? 

-  No.  [sorry,  I  did  not  ask  for  elaboration] 

Increased  working  speed? 

-Yes. 

Coordination  with  inbound  ground  controller? 

-  No,  because  there  was  no  inbound  traffic  to  coordinate. 

Flight  strip  marking  and  stripboard  management? 

-  It  provided  good  help,  although  more  work  is  needed  to  get  the  instructor,  student,  and  simulation 
working  at  the  same  speed. 

Phraseology  and  correct  frequencies? 

-  It  was  helpful  for  learning  to  use  correct  frequencies.  The  voice  recognition  was  too  picky,  failing 
to  accept  an  instruction  if  the  student  said  "the"  instead  of  "a",  spoke  too  loud,  soft,  or  fast.  It 
distracted  from  what  the  student  was  trying  to  learn. 

Dealing  with  similar  callsigns? 

-  It  never  came  up. 

Avoiding  unnecessary  verbiage? 

-  The  training  helped,  but  not  the  simulation's  voice  recognition. 

Any  other  aspects  of  your  present  work  with  actual  traffic  where  you  are  finding  that  the  simulation 
helped? 

-  The  location  of  gates  and  where  to  look  for  airplanes  calling  to  taxi  out.  It  sped  up  the  learning 
process. 

-  "I  can't  really  put  it  in  numbers,  but  I  feel  a  lot  better  about  approaching  the  ground  control 
learning  after  a  month  here....  I  feel  well  ahead  of  the  game  here." 

-  "I  think  it  was  very  beneficial...!  think  it  helped  in  all  areas...it  greatly  helped  with  stripboard 
management,  stripmarking,  and  phraeology,  not  from  voice  recognition,  but  from  practice  and 
repetition,  saying  the  routes  over  and  over. 
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How  did  the  speed  of  the  simulation  compare  with  actual  traffic? 

-  The  speed  of  the  simulation  problems  was  in  same  range  over  slow  periods,  but  it  did  not  get  as 
fast  as  it  does  upstairs;  the  7:30  rush  does  not  compare  to  the  simulation. 

-  It  is  hard  to  compare  without  inbounds  in  the  simulation  because  inbounds  can  become  confused 
with  your  [outbound]  traffic  (which  adds  complexity).  "You  are  looking  at  more  than  your  guys 
upstairs;  you  are  looking  at  everything." 

What  improvements  would  be  most  important? 

-  A  360  degree  screen  would  help  the  most. 

-  Simulation  operators  to  run  remote  positions;  one  to  run  inbounds,  another  to  be  the  pilot  for  the 
outbounds,  and  then  there  would  be  the  instructor,  watching.  You  could  set  up  traps  where  certain 
pilots  miss  their  instructions. 

-  Voice  recognition  would  be  valuable  if  it  allowed  a  student  to  practice  without  imposing  on 
others'  time.  This  would  allow  for  the  additional  practice  needed  to  develop  habits  which  in  turn 
would  translate  to  time  saved.  Four  weeks  were  not  enough  to  develop  stripmarking  and  stripboard 
management  skill  habits;  "I  didn't  get  to  writing  the  number  on  the  strip,  looking  out  the  window 
and  moving  it,  all  without  thinking.  I  still  had  to  think  about  writing  the  number  on  the  strip, 
moving  the  strip,  and  sequencing  it,  and  figure  out  which  one's  an  American  and  which  one's  a 
United  and  putting  them  between  those  two.  The  habits  didn't  come  for  me  as  fast  as  they  said  they 
should  because  if  you  do  it  only  one  hour  per  day  for  20  days...." 
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APPENDIX  B:  BASELINE  DEVELOPMENT  OF  GROUND  CONTROL  SKILLS 


“Average  Learner” 


Ground  Control  Skill 

1 

wk 

1 

mo 

2 

mo 

M 

6 

mo 

Visually  scans  airport  surface 

2.0 

2.0 

2.7 

3.0 

5.0 

1  “Never  2=Seldom  3=Often  4= Almost  Always  5=Always 


Maintains  efficient  traffic  flow 

1.3 

2.0 

2.0 

2.7 

1  “Never  2=Seldom 

3=Often 

4“Almost  Always 

5  “Always 

Maintains  aircraft  identity 

1.7 

2.0 

3.0 

3.0 

4.0 

4.7 

4.7 

1  “Never  2=Seldom 

3“Often 

4“ Almost  Always 

5“Always 

Effectively  manages  stripboard 

1.3 

1.7 

2.3 

2.7 

3.7 

4.0 

4.7 

5.0 

1  “Never  2“Seldom 

3=Often 

4“Almost  Always 

5“Always 

Working  speed 

1.3 

1.3 

1 

2.7 

3.3 

4.3 

4.7 

5.0 

l=Very  Slow  2=Adequate  for  Low  Workload  3=Adequate  for  Moderate  Workload 
4=Adequate  for  High  Workload  5=Adequate  for  Very  High  Workload 


1  Missed/delayed  traffic  calls 

1.0 

1.0 

1.3 

2.7 

3.3 

4.3 

4.7 

l“Many  2=Some  3“Few 

4“ Almost  None 

5= 

=None 

Communication  is  clear  &  concise 

1.7 

1.7 

2.3 

3.3 

4.0 

Ea 

1  “Never  2“Seldom  3=Often 

4“ Almost  Always 

5  “Always 

Makes  unnecessary  transmissions 

1.0 

1.0 

1.7 

2.3 

3.7 

4.0 

4.3 

4.7 

l=Many  2=Some  3=Few  4= Almost  None  5=None 
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“Fast  Learner” 


Ground  Control  Skill 

1 

wk 

12  4  6 

mo  mo  mo  mo 

8  10  12 

mo  mo  mo 

Visually  scans  airport  surface 

2.0 

2.7  3.7  4.0  4.7 

5.0  5.0  5.0 

l=Never  2=Seldom 

3=Often 

4=Almost  Always 

5=Always 

Maintains  efficient  traffic  flow 

1.3 

1.7  2.7  3.3  4.3 

5.0  5.0  5.0 

1  =N  ever  2=Seldom 

3=Often 

4= Almost  Always 

5=Always 

Maintains  aircraft  identity 

2.3 

1  =N  ever  2=Seldom 

3=Often 

4=Almost  Always 

5=Always 

Effectively  manages  stripboard 

2.0 

2.0  3.0  3.7  4.7 

5.0  5.0  5.0 

l=Never  2=Seldom 

3=Often 

4=Almost  Always 

5=Always 

Working  speed 

2.0 

2.7  2.7  3.7  4.0 

4.7  5.0  5.0 

l=Very  Slow  2= Adequate  for  Low  Workload  3=Adequate  for  Moderate  Workload 
4=Adequate  for  High  Workload  5=Adequate  for  Very  High  Workload 


1  Missed/delayed  traffic  calls 

1.3 

1.7 

2.7 

3.3 

4.0 

4.3 

4.7 

4.7 

l=Many  2=Some  3=Few 

4= Almost  None 

5= 

=None 

Communication  is  clear  &  concise 

2.3 

3.0 

4.7 

5.0  1 

l=Never  2=Seldom  3=Often 

4=Almost  Always 

5=Always 

Makes  unnecessary  transmissions 

2.0 

2.0 

2.7 

3.7 

4.3 

4.7 

4.7 

5.0  1 

l=Many  2=Some  3=Few  4= Almost  None  5=None 
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APPENDIX  C:  OUTLINE  OF  RECOMMENDED  REQUIREMENTS  FOR  A  TOWER 

SYSTEM  ASSESSMENT  SIMULATOR 


A.  Display 

1 .  Room  space  sufficient  to  replicate  largest  tower 

2.  Moveable  realistic-size  window  displays  each  showing  the  view  from 

one  side  of  the  actual  tower 

3.  All  tower  window  views  displayed  simultaneously 

4.  Photographic  color  images  of  entire  airports,  readily  updatable 

5.  Color  photographic  images  of  aircraft  of  types  and  companies  appropriate  to 

airport,  readily  updatable 

6.  Inbound  and  outbound  aircraft  in  local  controller  airspace  at  realistic  speeds 

which  vary  with  aircraft  type 

7.  Aircraft  taxiing  in  and  out  at  realistic  speeds  which  vary  with  aircraft  type 

8.  Capable  of  exchanging  the  photograph  of  one  airport  for  another  airport 

9.  Maximum  volume  of  aircraft  equal  to  future  enplanements  predicted  for  simulated  airport 

10.  Aircraft  and  gates  individually  scheduled 

1 1 .  Ability  to  remove,  modify,  and  rename  current  taxiways  and  runways  and  to  create  new  or 
extended  taxiways  and  runways 

12.  Sessions  recorded  for  playback 

B.  Control  (by  human  operator  or  automated  system) 

1 .  Individual  aircraft  movements  along  specified  taxiways  and  runways 

2.  Takeoff/landing  roll  and  liftoff/landing  at  realistic  runway  positions  and  speeds 

3.  Verbal  responses  transmitted  through  controller  headsets 

C.  Positions  (include  furniture  and  devices) 

1 .  ATC  (8):  two  ground  control,  two  local  control,  flow  manager,  flight  data,  clearance 

delivery,  supervisor,  and 

2.  Simulation  operator/pilot  positions  (4);  two  respond  to  local  control;  two  respond  to  ground 

control,  or 

3.  Automated  speech  recognition/synthetic  pilot  and  one  simulation  operator  who  can 

override  the  automated  pilot  systems 

D.  Interface  to  systems  under  test 

1 .  Actual  current  and  new  system  user  interface 

2.  Capable  of  simulating  inputs  to  current  or  new  systems 

3.  Capable  of  simulating  faults  in  new  systems 

4.  Adaptable  to  new  automation  system  interfaces 

5.  Quickly  reconfigurable  with  different  combinations  of  equipment 

E.  Interface  to  controller  performance  data  logging  equipment 

1 .  Source  of  input  (device  used) 

2.  Identity  of  data  or  command  correctly  or  incorrectly  entered 

3.  Time-logging  of  all  data  collected,  synchronized  to  simulation 
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F.  Additional  data  collection 

1 .  Videotape  from  each  window  display  location,  synchronized  to  simulation 

2.  Audiotape  from  each  control  position,  synchronized  to  simulation 

3.  Record  of  simulation  operator  commands  and  time  entered 
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